ec07

50
Vaccine efficacy (EC07) Module: EPM301 Epidemiology of Communicable Diseases Course: PG Diploma/ MSc Epidemiology This document contains a copy of the study material located within the computer assisted learning (CAL) session. The first three columns designate which page, card and screen position the text refers to. If you have any questions regarding this document or your course, please contact DLsupport via [email protected] . Important note: this document does not replace the CAL material found on your module CDROM. When studying this session, please ensure you work through the CDROM material first. This document can then be used for revision purposes to refer back to specific sessions. These study materials have been prepared by the London School of Hygiene & Tropical Medicine as part of the PG Diploma/MSc Epidemiology distance learning course. This material is not licensed either for resale or further copying. © London School of Hygiene & Tropical Medicine September 2013 v1.0

description

stats notes

Transcript of ec07

  • Vaccine efficacy (EC07)

    Module: EPM301 Epidemiology of Communicable Diseases

    Course: PG Diploma/ MSc Epidemiology

    This document contains a copy of the study material located within the computer assisted learning (CAL) session. The first three columns designate which page, card and screen position the text refers to. If you have any questions regarding this document or your course, please contact DLsupport via [email protected]. Important note: this document does not replace the CAL material found on your module CDROM. When studying this session, please ensure you work through the CDROM material first. This document can then be used for revision purposes to refer back to specific sessions. These study materials have been prepared by the London School of Hygiene & Tropical Medicine as part of the PG Diploma/MSc Epidemiology distance learning course. This material is not licensed either for resale or further copying.

    London School of Hygiene & Tropical Medicine September 2013 v1.0

  • Section 1: EC07 Vaccine efficacy Aims

    To learn about the issues involved in assessing vaccine efficacy Objectives By the end of this session you should be able to:

    describe the concept of vaccine efficacy list the epidemiological methods that can be used to estimate vaccine

    efficacy identify important methodological issues in designing studies to estimate

    vaccine efficacy explain the meaning of direct and indirect effects of vaccination distinguish between efficacy and effectiveness compare the proposed mechanisms underlying vaccine protection

    This session should take you between 2.5 and 4 hours to complete. Section 2: Introduction what is vaccine efficacy? Immunity and vaccination are concepts of infectious disease epidemiology. Natural or artificially induced immunity may protect an individual against an infectious agent, or against the disease associated with the infection. For non-infectious diseases, it is rare that individuals are exposed to the cause and yet are protected from the disease. The exception to this is through the mechanism of genetic susceptibility or resistance, but this cannot be artificially induced.

    Vaccination is one of the major public health intervention tools available against infectious diseases. In this session you will be introduced to some important issues relating to vaccine efficacy.

    As vaccines do not always work, you need to be able to distinguish between:

    vaccination: the administration of an antigen with the intention of inducing a specific immune response against an infectious agent or toxin, and

    immunisation: the successful artificial induction of effective,

    protective immunity against infection and/or disease. 2.1: Introduction what is vaccine efficacy? The standard measure of how well a vaccine works is vaccine efficacy (VE). This is defined as "the percent reduction in incidence among vaccinated individuals, which is attributable to vaccination", and can be calculated as:

    Vaccine efficacy (VE) = lu lv x 100 lu where:

  • Iu = incidence in unvaccinated individuals Iv = incidence in vaccinated individuals

    If the vaccine gives total protection there will be no cases in the vaccinated group and VE will be 100%. If the vaccine gives no protection, the incidence will be the same in both groups and VE will be 0%. You can rearrange the formula to show how vaccine efficacy is related to the relative risk (rate ratio, risk ratio or odds ratio) of the disease in vaccinated compared to unvaccinated individuals (i.e.those not vaccinated with the vaccine of interest):

    Vaccine efficacy (VE) = Iu Iv x 100 Iu Iu VE = (1 Relative Risk) x 100

    Refer back to Session FE05 to review measures of relative risk. 2.2: Introduction what is vaccine efficacy? Some vaccines act to prevent infection while others prevent the progression from infection to disease.

    Vaccine efficacy is generally measured in relation to the incidence of infection or different grades of disease severity, depending on the mechanism through which the vaccine is thought to work.

    Some studies use seroprevalence or seroconversion as indicators. However these are dependent on the relationship between such immunological measures and protective immunity. Interaction: Hyperlink: seroprevalence: Seroprevalence The percentage of the population that are seropositive (produce antibodies in response to a specific antigen) Interaction: Hyperlink: seroconversion: Seroconversion The change from being seronegative (producing no antibodies in response to a specific antigen) to being seropositive (producing antibodies in response to that specific antigen). It requires two tests on the same individual in order to detect any change in serological status. Interaction: Tabs: Seroconversion: In seroconversion studies, individuals are surveyed before vaccination (to exclude those already immune) and shortly after vaccination, to measure the percentage of individuals who change serological status.

  • This can show whether a vaccinated individual has produced an antibody response to the antigens included in the vaccine, that is, whether the individual has been vaccinated.

    If there is an immunological marker that is a good correlate of protective immunity, this can be used to assess whether individuals have been immunised. Interaction: Hyperlink: immunological marker (pop up box appears): Immunological marker Immunological markers usually refer to the specific immune responses that can indicate experience of infection or vaccination. Interaction: Tabs: Seroprevalence: A seroprevalence survey is used to measure the percentage of the population that is seropositive. This is often difficult to interpret because antibodies may be due to maternal antibodies, natural infection before or after vaccination, or vaccination.

    Apparent seronegativity may be due to an absence of immunity, or to a fall in antibody levels below a detectable level. In the latter case, antibody levels could rise again in response to re-infection. Therefore the individual is still protected, although apparently seronegative. Section 3: Methods to estimate vaccine efficacy A number of epidemiological methods can be used to estimate vaccine efficacy, and these will be considered in more detail on the following pages.

    You may wish to refer to FE08 to review some of these study designs. 1. Randomised control trials 2. Cohort studies 3. Outbreak investigations 4. Contact studies 5. Retrospective cross-sectional studies 6. Case-control studies 7. Case-population studies Section 4: Method 1: Randomised controlled trial A randomised controlled trial (RCT) can be regarded as an experimental form of cohort study. It involves the random allocation of vaccine to some individuals, and a placebo or alternative existing intervention to others. These individuals are then followed-up over a period of time to identify cases of the disease.

    Ideally, neither the vaccine recipient nor the people assessing the outcome should know who has received which treatment. This is known as double-blinding and aims to prevent both the recipient and the assessor from biasing the outcome of the study.

  • Interaction: Hyperlink: placebo: Placebo An inactive intervention, administered to the control group in a controlled trial of an experimental treatment. The experimental treatment must produce better results than the placebo in order to be considered effective. Interaction: Hyperlink: double-blinding: Blinding (also know as masking) Concealment of information about exposure or outcome in order to reduce bias. In a single-blind clinical trial, the participants do not know which treatment they have been assigned (but the investigators do know). In a double-blind trial, neither participants nor investigators know which treatment has been allocated to which patient. In a triple-blind trial, in addition to the participants and the investigators, the people performing the data analysis are also unaware of which group received what treatment so their analysis is not influenced by their expectations of which treatment is more effective. The incidence in each group can then be used to calculate the vaccine efficacy. In this case the measure of incidence will be the rate or the risk of disease.

    RCTs are considered the best method to measure vaccine efficacy because the randomisation controls for confounding effects (such as genetics, socioeconomics, etc.) and the 'blinding' reduces information biases. 4.1: Method 1: Randomised controlled trial Most vaccines are evaluated in one or more RCT before being licensed for general use.

    These trials are known as Phase 3 trials, as they are only carried-out after earlier studies have shown that the vaccine is safe in humans (Phase 1) and that it is effective in small numbers of volunteers (Phase 2).

    Phase 1 and 2 trials are usually not randomised and test the vaccine under ideal conditions. Phase 3 trials then assess efficacy in large numbers of individuals, under more realistic conditions. Interaction: Hyperlink: ideal conditions: For example, the study individuals are usually healthy adult volunteers, chosen because they are more likely to respond to the vaccine, and the efficacy is then tested by artificially inducing infection in these volunteers. 4.2: Method 1: Randomised controlled trial RCTs may compare a new vaccine with a placebo and this will provide a measure of absolute efficacy.

  • Alternatively, if a vaccine currently exists, the new vaccine should be compared with the existing vaccine (for ethical reasons) to provide a measure of relative efficacy.

    Ethical issues are becoming increasingly prominent in intervention trials. It is important that any research study considers what is best for all the study participants during the study, in addition to the potential benefits that may result from the research itself. Ethics

    If a partially useful vaccine or intervention is already available, it is unethical to have a placebo group that receives no protection from the infection or disease. In the same way, if the current vaccine is quite good and the disease is relatively rare, it may not be considered ethical to set-up a RCT to evaluate a new vaccine.

    However, if no vaccine is currently available, a vaccine against another infectious agent (that has no effect on the disease of interest) is often given to the 'control' group in the vaccine trial. This ensures that this group also benefits in some way from participating in the trial. 4.3: Method 1: Randomised controlled trial Exercise

    A randomised double-blind placebo-controlled trial of the malaria vaccine SPf66, was undertaken to assess its efficacy and utility in an area of high malaria transmission in southern Tanzania. In the trial, children aged 1-5 were studied, as this is the age group most at risk of malaria disease in this setting.

    Read the abstract, introduction and methodology of the paper reporting the results of this trial (Alonso et al.) in your workbook and answer the following questions.

    In any intervention trial it is necessary to first define the end-point being considered, i.e. is the intervention aimed at reducing infection, disease or death?

    What was the main outcome of interest in this study? Interaction: Button: clouds picture (pop up box appears): The main outcome of interest was the incidence of clinical malaria with the Plasmodium falciparum species, defined as measured fever at or above 37.5oC and parasite density above 20,000 parasites per microlite (l) o f b lo o d . Th e other outcome of interest was the incidence and density of parasitaemia (see Summary and Study Design in Alonso et al.) This means that the vaccine was tested for its effect against disease, but also its effect in reducing detectable infection. Interaction: Hyperlink: parasitaemia: Parasitaemia

  • Refers to the presence of parasites (i.e. infection), and also to the amount of parasites present (i.e. density). 4.4: Method 1: Randomised controlled trial What was the vaccine compared to?

    Hint: See the section titled 'Vaccine' in the Alonso et al. paper. Interaction: Button: clouds picture (text appears on bottom LHS and card appears on RHS): The vaccine was compared to a placebo, as no malaria vaccine is currently available. The placebo was aluminium hydroxide given with tetanus toxoid in the first dose. Interaction: Hyperlink: aluminium hydroxide: Aluminium hydroxide This is included in some vaccines to stimulate the immune response towards antibody production. Interaction: Hyperlink: tetanus toxoid: Tetanus toxoid This vaccinates against tetanus but has no known effect against malaria, and provides some benefit to the placebo group. The vaccine and placebo were bottled identically with numerical codes to ensure that the trial was 'double-blind', that is, neither the study investigators nor the study participants were aware of which treatment they had received. The codes were revealed after the study had been completed and the analysis had been agreed upon. 4.5: Method 1: Randomised controlled trial Look at the results shown in Table 4 of Alonso et al.

    Because this was a prospective study, the observation time was recorded and the measure of incidence obtained was an annual rate of disease. Individuals can suffer multiple episodes of clinical malaria, so for simplicity only the first episode detected during the trial was used in the analysis.

    The vaccine efficacies reported in Table 4 have been rounded up to the nearest percentage point. Calculate the unadjusted vaccine efficacy after 3 doses of vaccine (2nd row of Table 4) to one decimal place and enter your answer in the box below.

    Remember that VE = (lu _lv) / lu x 100, where Iu represents the incidence in the unvaccinated individuals and Iv represents the incidence in the vaccinated individuals.

    Hint: Look carefully at the units in the table.

  • Unadjusted VE =

    You can read the full paper later in this session to see how these results were interpreted. Interaction: Calculation: Unadjusted VE = : Correct Response 28.3: Correct Thats right. The annual incidence rate in vaccinated children is: lv = (49 / 72052) x 365 = 0.248, and the annual incidence rate in unvaccinated children is: lu = (74 / 78046) x 365 = 0.346. So VE = (0.346 0.248) / 0.346 x 100 = 28.3%. (The time at risk was given in child-days, this needed to be adjusted in the calculation of an annual rate). Incorrect Response: Thats not right. The annual incidence rate in vaccinated children is: lv = (49 / 72052) x 365 = 0.248, and the annual incidence rate in unvaccinated children is: lu = (74 / 78046) x 365 = 0.346. So, VE = (0.346 0.248) / 0.346 x 100 = 28.3%. (The time at risk was given in child-days, this needed to be adjusted in the calculation of an annual rate). Section 5: Method 2: Cohort Studies Once vaccines are in use, populations with known vaccination status can be followed-up to assess the vaccine efficacy in a natural setting. This can be done using clinic records or 'road-to-health' cards from routine vaccination programmes if the vaccination status is well recorded for all individuals, or if it can be inferred as with the distinct scar resulting from BCG vaccine administration.

    Incidence measures can be calculated from disease surveillance during the follow-up period, and vaccine efficacy can be calculated from the comparison of vaccinated and unvaccinated cohorts. However, vaccines administered through routine programmes may not be randomly allocated. Not everybody gets equal access to healthcare, as some individuals do not visit the health centres or attend school.

    Can you think why this might bias the results? Interaction: Button: clouds picture (pop up box appears): Individuals who do not get vaccinated may also have a different risk of the disease of interest. For example, a child may not attend school for economic reasons, and their lower socio-economic status may predispose them to have a higher risk of contracting measles. Therefore a higher incidence of disease among unvaccinated individuals may be due to confounding factors associated with allocation of the vaccine, rather than

  • due to an effective vaccine. It is therefore important to account for confounding in the study design and analysis. Interaction: Hyperlink: confounding (pop up box appears): Confounding Confounding is about alternative explanations for an association between an exposure and an outcome. Confounding occurs when there is unequal distribution of a risk factor for a disease between those exposed and unexposed to the exposure of interest. In order to be a confounder, a variable must satisfy two conditions:

    it must be associated with the exposure of interest it must be a risk factor for the outcome of interest

    Section 6: Method 3: Outbreak investigations Outbreak investigations provide an attractive opportunity for studying vaccine efficacy. The fact that cases are clustered in space and time makes it easier to collect sufficient data. In such a setting it is also more likely that the exposure to infection will be similar in all individuals, minimising the confounding effects.

    The population at risk is classified into vaccinated and unvaccinated individuals according to their vaccination status at the beginning of the outbreak. Attack risk can then be measured for each group, and the vaccine efficacy can be calculated. Interaction: Hyperlink: Attack risk: Attack rate An attack rate is a measure of risk. It is defined as the number of new cases occurring during the duration of an outbreak among the population at risk at the start of the outbreak. The attack rate is a measure of risk. It is defined as the number of new cases occurring during the duration of an outbreak among the population at risk at the start of the outbreak. The attack rate is measured over a short period of time the duration of the outbreak or epidemic and the incidence is therefore higher than usual. Limitations Interaction: Tabs: Local efficacy: A potential limitation of this approach is that VE estimated during an outbreak may reflect the local efficacy, and may not be generalised to other contexts. Outbreaks are often exceptions, and may not be representative of the entire population.

    For example, an outbreak may reflect a local problem, such as a poor batch of vaccine or a chance cluster of vaccination failures. Interaction: Tabs: Failures:

  • A vaccination failure occurs when a vaccination event fails to produce protective immunity (immunise). This may be due to a variety of factors such as poor vaccine production, problems with delivery or handling, inappropriate administration, host factors or environmental factors.

    Clustering of such susceptible individuals also isolates them from indirect protection by immune members of the population ('herd immunity'). This tends to lead to an underestimate of VE in outbreak investigations (Fine & Zell 1994). Interaction: Hyperlink: delivery or handling(pop up box appears): For example, a breakdown of the cold chain. Interaction: Hyperlink: inappropriate administration: (pop up box appears): For example, the wrong dosage injected. Interaction: Hyperlink: host factors (pop up box appears): For example, interference of maternal antibodies. Interaction: Hyperlink: environmental factors (pop up box appears): For example, a variant form of the pathogen. 6.1: Method 3: Outbreak investigations The following table shows the number of measles cases among vaccinated and unvaccinated children aged 9 47 months, from an outbreak investigation in the Gambia (Hull et al. 1983).

    Using the attack rates (AR) as a measure of disease incidence, calculate the efficacy of the measles vaccine in this setting. Enter your answer as a percentage to one decimal place in the box below.

    Vaccine efficacy = % Interaction: Calculation: Vaccine efficacy = % Correct Response (pop up box appears and text appears on bottom RHS): Correct Yes, the vaccine efficacy is 87.0%, calculated as follows: VE = (ARu ARv) x 100 ARu

  • = (43.0 -5.6) x 100 43.0 = 87.0%. Now look at Table III in the paper by Hull et al. (1983). The vaccine efficacy you have calculated here is an average, but in the paper the authors calculate vaccine efficacy according to the age at vaccination. Which is more appropriate?

    Interaction: Button: clouds picture (pop up box appears): It is important to consider the best age at which to vaccinate, because the presence of maternal antibodies can interfere with the development of an effective immune response to vaccination. Vaccine efficacy by age at vaccination therefore provides more information on the usefulness of the vaccine and identifies the most appropriate vaccination schedule.

    (You will get a chance to read the full paper later in this session to see how the outbreak investigation was conducted and what the authors concluded.) Section 7: Method 4: Contact studies This approach to assessment of vaccine efficacy is generally used for infections that are transmitted efficiently within a household, as for pertussis and tuberculosis. Index cases can be identified through routine disease notifications, and their households visited as soon as possible. Individuals living with the index cases are considered to be contacts and are classified according to vaccine status.

    The incidence of disease can then be obtained, either retrospectively or prospectively, for vaccinated and unvaccinated contacts, and the VE can then be calculated. Interaction: Hyperlink: pertussis: Pertussis Refer to http://www.cdc.gov/pertussis/ for more information about this disease. Interaction: Hyperlink: tuberculosis: Tuberculosis Refer to http://www.cdc.gov/tb/ for more information about this disease. If the infection has a short and discrete incubation period, the incidence rates should be calculated as household secondary attack rates (Review this from session EC03). It may also be possible to extend this approach to include neighbourhood contacts.

    One of the advantages of contact studies is that all the contacts should be equally exposed to the infection, independent of their vaccination status. However, there could be a bias in the selection of households, as families with a large number of

  • cases may be more likely to be recruited and could be more likely to include vaccination failures. 7.1: Method 4: Contact studies Exercise

    A household contact study was undertaken as part of a prospective, cohort vaccine efficacy trial against pertussis in Germany. Infants were vaccinated with one of two diphtheria-tetanus-pertussis (DTP) vaccines, and compared with a control group receiving only combination diphtheria and tetanus (DT) vaccine.

    Calculate the VE for the acellular (DTaP) and whole-cell (DTP) pertussis components using the secondary attack rates (SAR) shown, and fill-in the answers in the table opposite to the nearest percent.

    Interaction: Calculation: DTP/Vaccine Efficacy (%): Correct Response 80% (pop up box appears): Yes, the vaccine efficacy for DTIP is 80%. Using the formula: VE = 100 x (SARu SARv) / SARu, where SAR has been substituted as the appropriate measure of incidence in a contact study, the answer can be calculated as: VE = 100 x (47.1 9.2) / 47.1 = 80%. Incorrect Response (pop up box appears): Sorry, in fact the vaccine efficacy for DTP is 80%. You should have used the formula: VE = 100 x (SARu SARv) / SARu, where SAR has been substituted as the appropriate measure of incidence in a contact. In this way, the answer can be calculated as: VE = 100 x (47.1 9.2) / 47.1 = 80%. Interaction: Calculation: DTaP/Vaccine Efficacy (%): Correct Response 55 (pop up box appears): Yes, the vaccine efficacy for DTaP is 55%. Using the formula: VE = 100 x (SARu SARv) / SARu, where SAR has been substituted as the appropriate measure of incidence in a contact study, the answer can be calculated as: VE = 100 x (47.1 21.3) / 47.1 = 55%. Incorrect Response (pop up box appears):

  • Sorry, in fact the vaccine efficacy for DTaP is 55%. You should have used the formula: VE = 100 x (SARu SARv) / SARu, where SAR has been substituted as the appropriate measure of incidence in a contact study. Then the answer can be calculated as: VE = 100 x (47.1 21.3) / 47.1 = 55%. 7.2: Method 4: Contact studies Now read the abstract, introduction and methodology sections in the paper describing this study in your workbook (Heininger et al. 1998).

    A number of different case definitions were used to identify primary and secondary cases. Is the following statement true or false?

    "All family members with cough onset within 7 days of illness onset of the primary case were considered to be secondary cases."

    Interaction: Hotspot: True: Incorrect Response (pop up box appears): In fact, household contacts with onset within 7 days of the primary case were considered to be co-primary cases (see Case Definitions in Heininger et al.) as the incubation period is commonly 7 20 days. Interaction: Hotspot False: Correct Response (pop up box appears): Correct Thats right. Household contacts with onset within 7 days of the primary case were actually considered to be co-primary cases (see Case Definitions in Heininger et al.) as the incubation period is commonly 7-20 days. It has been reported that household contact studies provide a lower estimate of VE for pertussis vaccines than cohort studies. Fine et al. (1988) suggested that this discrepancy may be due to the inclusion of retrospectively assessed cases and of households in which the primary case was a vaccine failure.

    How did this study address these issues? Write down your answer and then check the feedback below. Interaction: Button: clouds picture (pop up box appears): All cases in this study were obtained prospectively by active surveillance and parent reporting (se Serum Collection and Case Ascertainment in the paper). In addition, cases included in the principal efficacy analysis were laboratory

  • confirmed, removing the problem of misdiagnosed secondary cases. Primary case vaccine failures were removed, as no vaccines that were primary or co-primary cases were included in the analysis (see Case Definitions in the paper).

    You will get a chance to read this paper later in this session and consider the main issues involved in outbreak investigations. Section 8: Method 5: Retrospective cross-sectional studies If a disease is relatively common and easily diagnosed, it is possible to obtain retrospective information on the incidence of the disease in a cross-sectional survey.

    The Expanded Programme on Immunisations (EPI) has developed a standard vaccine coverage survey protocol, according to which information on vaccine uptake is collected. This is known as 'cluster sampling' as vaccination status is determined by examining clinic records or 'road-to-health' cards on 210 children distributed in 30 clusters (7 children per cluster). Disease history is determined up to the current age of the children in the survey. Because the interval from disease to the time of survey may be quite long, a history is taken from parents and is used to identify and define cases.

    However, many diseases are often not sufficiently distinctive, as symptoms may not be specific to only one disease. It is therefore important to validate the diagnosis, and this can be a major limiting factor to this approach.

    Individuals with uncertain vaccination or disease histories should be considered separately in the analyses. 8.1: Method 5: Retrospective cross-sectional studies Measles vaccine efficacy was estimated in this way by asking mothers about the history of measles disease in their children, including dates of onset. Read the abstract, introduction and methodology sections from the paper by Cutts et al. (1990).

    The equation shown in the introduction is a quick method of estimating vaccine efficacy and will be introduced in this session under the section on retrospective cohort studies. Why were cases only included if they occurred after 12 months of age? Interaction: Button: clouds picture (pop up box appears and text appears on the bottom RHS): Only children vaccinated between 9-11 months of age and those never vaccinated, were included in the study. If cases occurring before 12 months were included, this might have included cases in vaccinated children that had occurred prior to vaccination. This would have underestimated the vaccine efficacy. Now read the Results section of the paper. Try calculating the vaccine efficacy for one of the studies from the data in Table 1 of the paper. Do you get the same result as the authors?

  • Interaction: Button: clouds picture: The vaccine efficacy estimates presented for each study in Table 1 are inconsistent with a simple analysis of the data presented. This is because the VE estimates have been adjusted for the effect of cluster sampling, and for the effect of misdiagnosis see next card. 8.2: Method 5: Retrospective cross-sectional studies To test the validity of the maternal reporting, a survey was conducted to measure the seroprevalence of measles antibodies in the unvaccinated study population.

    Using the serologic findings, this study estimated the specificity of measles diagnosis by the mother as being 83%. This means that out of the 64 children who were seronegative, only 53 (83%) were reported as never having had measles. Eleven children (17%) were reported as having had measles, but had no serological evidence of previous infection, implying that the mother's reporting of measles history was incorrect. Interaction: Hyperlink: specificity (pop up box appears): Specificity The specificity of a test is the proportion of true negatives correctly identified as such by the test. This is different fro the negative predictive value, which is the proportion of identified negatives that truly are negative. Specificity is equal to 1 minus the proportion of false positives those true negatives that are incorrectly identified as positive. Interaction: Hyperlink: seronegative: Seronegative Seronegative means that the individual or sample tested did not have specific antibodies to the antigens tested. Interaction: Tabs: Adjusted VE: Using these data, the estimated vaccine efficacy was adjusted by this serologic finding to account for any misdiagnosis by the mothers

    A positive predictive value and a negative predictive value were calculated from the unvaccinated group, and this was used to estimate the number of true positives and negatives among the vaccinated group. Interaction: Hyperlink: positive predictive value: Positive Predictive Value The proportion of individuals with a positive test result who genuinely are positive.

  • This is different from sensitivity, which is the proportion of true positives that are correctly identified as such. Also known as predictive value positive. Interaction: Hyperlink: negative predictive value (pop up box appears): Negative Predictive Value The proportion of individuals with a negative test result who genuinely are negative. This is different from the specificity, which is the proportion of true negatives that are correctly identified as such. Also known as predictive value negative. Interaction: Tabs Exercise: Using this adjustment, how many of the children that were vaccinated seroconverted as a result of vaccination?

    Number of seroconversions = The implications of case misclassification will be further discussed later in this session. (You will be given an opportunity to read the remainder of the paper later in this session. Use that reading to consider the benefits and deficits of this methodology.)

    Interaction: Calculation: Number of seroconversions =___: Correct Response 161 (pop up box appears): Correct The authors estimated that 12 seropositives with a history of measles were true positives, 48 seropositives with no history of measles were also true positives. Out of the 221 seropositives in the vaccinated group, that leaves only 161 children (66%) in whom seropositivity was induced by vaccination (see Results section). Incorrect Response (pop up box appears): The authors estimated that 12 seropositives with a history of measles were true positives, 48 seropositives with no history of measles were also true positives. Out of the 221 seropositives in the vaccinated group, that leaves only 161 children (66%) in whom seropositivity was induced by vaccination (see Results section).

  • Section 9: Method 6: Case-control studies The aim of a case-control study is to compare the relative proportions of vaccinated individuals among the cases and the controls. To estimate VE, cases of the disease are identified from routine surveillance or hospital records. 'Appropriate' control individuals then need to be selected, and vaccination status needs to be determined for all the individuals. By definition a case-control study focuses on detecting cases, and cannot provide an estimate of the actual disease incidence in the vaccinated and unvaccinated groups. For this reason, the initial VE formula (VE = [(Iu _Iv) / Iu] x 100) cannot be used. However, as you saw earlier, the VE can also be estimated from the relative risk. The odds ratio (OR) is frequently used to describe measures of effect in case-control studies and is an estimate of the relative risk. So the VE can be determined using:

    VE = 1 - OR

    If all cases were unvaccinated and all controls were vaccinated, the VE would be 100%. 9.1: Method 6: Case-control studies As with the other study designs, it is important to avoid confounding and bias when estimating vaccine efficacy.

    Again, there may be some bias associated with the probability of having been vaccinated. The association between vaccination status and the disease could be confounded by other factors, and this needs to be taken into account in estimations of vaccine efficacy.

    Although it is not always possible to identify individuals that were already infected or had the disease before they were vaccinated, efforts should be made to exclude such individuals from the study. The controls must be representative of the population from which the cases were drawn to avoid selection bias.

    To minimise confounding it is sometimes useful to use a matched study design. For example, for a case diagnosed at the age of 5 years, an age-matched study would need to determine the vaccination status of a control individual when they were also 5 years, that is at the same age of onset of disease in the case. 9.2: Method 6: Case-control studies One year after a campaign of mass vaccination against meningococcal disease, a case-control study was undertaken to assess the efficacy of the vaccine used in Rio de Janeiro, Brazil. Read the abstract, introduction and methodology sections in the paper by Noronha et al. (1995).

    How did the investigators exclude cases that had been vaccinated after being infected?

  • Hint: Find-out the incubation period for meningococcal disease (MD) from http://www.cdc.gov/meningitis/index.html . Interaction: Hyperlink: meningococcal disease (pop up box appears): Meningococcal disease For more information on this disease see http://www.cdc.gov/meningitis/index.html Interaction: Button: clouds picture (pop up box appears and text appears on upper RHS): The incubation period of Neisseria meningiditis varies between 2-10 days. By starting recruitment of new cases of MD 4 weeks after the administration of the second dose of the campaign (see Study Design in the paper), individuals infected prior to vaccination were excluded from the case definition. Were cases and controls likely to be comparable? (Consider how the controls were selected and defined.) Yes No Interaction: Hotspot: Yes: Correct Response (pop up box appears and text appears on bottom RHS): Correct Both cases and controls were recruited from hospitals where they had been admitted with different forms of meningitis. The probability of being vaccinated is associated with routine use of public health services, and this could be associated with other risk factors for the disease such as socio-economic status. However, as many of the causes of meningitis in the controls would also be associated with the same risk factors, this is unlikely to be a major bias in this study. As the symptoms for both cases and controls would be similar, it is likely that they would be comparable with regard to their use of health facilities when sick. It is also likely that they would come from the same hospital catchment areas. Although the study was not matched by hospital, the analyses were stratified by place of residence. Interaction: Hotspot: No: Incorrect Response (pop up box appears and text appears on bottom RHS): No, in fact the cases and controls were comparable, because they were both recruited from hospitals where they had been admitted with different forms of meningitis. The probability of being vaccinated is associated with routine use of public health services, and this could be associated with other risk factors for the disease such as socio-economic status. However, as many of the causes of meningitis in the controls would also be associated with the same risk factors, this is unlikely to be a major bias in this study.

  • As the symptoms for both cases and controls would be similar, it is likely that they would be comparable with regard to their use of health facilities when sick. It is also likely that they would come from the same hospital catchment areas. Although the study was not matched by hospital, the analyses were stratified by place of residence. 9.3: Method 6: Case-control studies Was the BC meningococcal vaccine effective in Rio de Janeiro? Select the appropriate answer from those given below. Interaction: Hotspot: No, the vaccine was not significantly protective in this setting: Incorrect Response (pop up box appears): In fact, the vaccine *was* significantly protective against all cases of meningitis and against serotype B in children aged 4 and above at the time of the first dose of vaccination (see Table 3 and Discussion in the paper). Interaction: Hotspot: Yes, the vaccine was significantly protective in all age groups.: Incorrect Response (pop up box appears): Thats incorrect. The vaccine was significantly protective against all cases of meningitis and against serotype B, but only in children aged 4 and above at the time of the first does of vaccination (see table 3 and Discussion in the paper). Interaction: Hotspot: Yes, the vaccine was significantly protective in children over 4 years of age.: Correct Response (pop up box appears): Thats correct. The vaccine was significantly protective against all cases of meningitis and against serotype B in children aged 4 and above at the time of the first dose of vaccination (see Table 2 and Discussion in the paper). This vaccine had already been tested in Cuba, so why was it important to estimate the vaccine efficacy in this setting? Interaction: Button: clouds picture (pop up box appears):

    No, the vaccine was not significantly protective in this setting

    Yes, the vaccine was significantly protective in children over 4 years of age.

    Yes, the vaccine was significantly protective in all age groups.

  • The vaccine had been tested in Cuba in a double-blind randomised control trial (see Introduction in the paper). This provides an estimate of how well the vaccine might work under controlled natural conditions. A case-control study assesses VE under normal programmatic conditions, which are likely to vary between countries. In addition, the study in Cuba had shown a high protective efficacy in all age groups (see p1056, paragraph 1), while a previous case-control study in Sao Paulo, Brazil had demonstrated a variable protection by age (see Introduction).

    Read the remainder of this paper after the session, noting the important issues involved in case-control studies. Section 10: Method 7: Case-population studies This is a variation of the case-control method, and uses population statistics on vaccine-uptake for the control group. It is a crude method that provides a preliminary estimation of VE by comparing the proportion of cases vaccinated with the proportion of the overall population vaccinated.

    However, with this method it is usually not possible to take into account the effect of confounding factors, except for age and sex. For instance, it is important to make sure that the cases and the comparison population used are of the same age group. In many situations the incidence among the vaccinated and unvaccinated will not be known with precision.

    The following formula (derived from the formula you have been using throughout this session) can also be used to estimate VE.

    PCV is the proportion of cases vaccinated and PPV is the proportion of the population vaccinated: VE = (PPV PCV) x 100 (PPV x [1-PCV]) 10.1: Method 7: Case-population studies The formula can be rearranged to give: PCV = (PPV VE x PPV) (1 VE x PPV) Note: This formula is calculating proportions, so VE should be used as a proportion rather than as a percentage. (Proportion = Percentage / 100)

    This is often called the 'rapid screening method', and can be used to reassure health workers that the proportion vaccinated among cases (PCV) is what would have been expected given the level of vaccine uptake (PPV) for an estimated vaccine efficacy. If vaccine efficacy is found to be lower than expected, detailed investigations should be conducted to determine the causes of vaccination failure.

  • Click below to see the graph generated from this formula, which shows the theoretical proportion of cases that will have been vaccinated in a given setting for different levels of vaccine efficacy. Interaction: Button: Graph (pop up box appears):

    You can see from the graph that, as vaccine coverage increases, a larger proportion of cases will occur among the vaccinated individuals. Source Orenstein WA, Bernier RH, Dondero TJ, Hinman AR, Marks JS, Bart KJ, Sirotkin B. Field evaluation of vaccine efficacy. Bulletin of the World Health Organisation 1985; 63(6): 1055-1068. Section 11: Meta-analysis As with any measure of effect, vaccine efficacy is often measured in more than one setting and using different methods. A meta-analysis is a systematic review of different studies. It is useful for getting a sense of the amount of variation in outcome, the reasons for variations and for estimating a summary measure across studies.

    The graph opposite shows the distribution of the protective effect of BCG vaccine against leprosy in 13 different studies. It is clear from this that although the VEs vary between 20 _ 81%, the vaccine provided protection in all studies. The bars represent 95% confidence limits around the estimate.

  • 11.1: Meta-analysis Look through the paper by Rodrigues et al. (1993) in your workbook. It describes a meta-analysis of BCG vaccine in relation to protection against tuberculosis.

    What was the hypothesis being tested? Choose the answer from the options given opposite. Interaction: Hyperlink: tuberculosis: Tuberculosis See http://www.cdc.gov/tb/ for more information about this disease. Interaction: Hotspot: Protective effect of BCG may vary according to the study design used.: Incorrect Response (pop up box appears): No, the hypothesis is that different immunological mechanisms may be associated with protective effect against different forms and sites of the disease. The study

    Protective effect of BCG may vary according to the study design used.

    Protective effect of BCG may vary according to the geographical location of the study.

    Protective effect of BCG may vary according to the form of the disease.

  • therefore aims to differentiate between the VE for pulmonary, meningeal and military forms of tuberculosis. Interaction: Hotspot: Protective effect of BCG may vary according to the geographical location of the study.: Incorrect Response (pop up box appears): Sorry, thats not correct. The hypothesis is that different immunological mechanisms may be associated with protective effect against different forms and sites of the disease. The study therefore aims to differentiate between the VE for pulmonary, meningeal and military forms of tuberculosis. Interaction: Hotspot: Protective effect of BCG may vary according to the form of the disease.: Correct Response (pop up box appears): Thats correct. The hypothesis is that different immunological mechanisms may be associated with protective effect against different forms and sites of the disease. The study therefore aims to differentiate between the VE for pulmonary, meningeal and miliary forms of tuberculosis. 11.2: Meta-analysis Now look at the VE estimates in Table 1 and Table 2 of the paper.

    What is the overall effect of BCG against meningeal and miliary tuberculosis? Write down your answer and then check the feedback below. Interaction: Button: clouds picture (pop up box appears): There were few randomised control trials (RCTs) that distinguished these forms of disease as an outcome of interest. The RCTs available showed a highly protective effect but the confidence intervals were very wide in all cases. The results from the case-control studies varied between 58-100%, but the confidence intervals were narrower. The summary protective effect was 75% with 95% confidence intervals between 61% and 84% (see footnote to Table 2). 11.3: Meta-analysis This meta-analysis allowed the authors to test their hypothesis from existing data, rather than needing to conduct a new trial. The summary estimates take into account the sample size of each study, and the results can be stratified by study design, geographical area, etc.

    However, there are a number of limitations to this method. Limitations Interaction: Tabs: Variation: Study designs usually vary quite a lot. While it may be possible to conduct different analyses for different types of studies, as in this case, sometimes there will not be so many studies conducted.

  • In addition, other factors will vary between the studies, such as age groups included, dosages given, geographical areas chosen. Interaction: Tabs: Publication bias: Another limitation is that meta-analyses are often conducted on published studies. However, there is a well-known publication bias, with studies with clear and positive results being more likely to be published.

    This would bias the overall effect, so it is important to include as many studies, both published and unpublished, in this type of analysis. Section 12: Methodological issues There are a number of methodological issues that relate to more than one type of study design, and these will be considered on this page. 1. Case definition and ascertainment 2. Comparability between groups 3. Time issues 1. Case definition and ascertainment Case definitions should be uniformly applied to all individuals in the study and need to be as specific as possible. Laboratory confirmation of some cases may help to demonstrate the accuracy of the case definition. You may wish to refer to session FE12 to review the issue of reliability. In general, low specificity lowers the estimate of vaccine efficacy, whereas low sensitivity will only decrease the power of the study to detect an effect of the vaccine. Interaction: Hyperlink: specificity: Specificity The specificity of a test is the proportion of true negatives correctly identified as such by the test. This is different from the negative predictive value, which is the proportion of identified negatives that are truly negative. Specificity is equal to 1 minus the proportion of false positives those true negatives that are incorrectly identified as positive. Interaction: Hyperlink: sensitivity: Sensitivity The sensitivity of a test is the proportion of true positives correctly identified as such by the test. This is different from the positive predictive value, which is the proportion of identified positives that truly are positive.

  • Sensitivity is equal to 1 minus the proportion of false negatives those true positives that are incorrectly identified as negative. 12.1: Methodological issues 1. Case definition and ascertainment As you have seen in the paper by Rodrigues et al. (1993), it is possible that a vaccine might confer different levels of protection against clinical disease, different forms of clinical disease, infection, infectiousness, etc. Some vaccines may not prevent the disease in all individuals, but may reduce the severity of disease among vaccinees who do contract an infection. This has been shown for measles, with vaccinated children requiring more intense exposure to become infected, having a lower mortality among secondary cases and being less infectious (Aaby et al., 1986). 12.2: Methodological issues

    1. Case definition and ascertainment

    It is therefore important that the outcome of interest is defined precisely, and the study should be designed with sufficient precision and power to measure protection against that outcome.

    You may wish to refer to Practical Epidemiology (EP103) session 6 to review the subject of sample sizes. Interaction: Hyperlink: precision: Precision In epidemiological studies, precision refers to the ability to measure the magnitude of effect with minimal sampling error. Interaction: Hyperlink: power: Power In epidemiological studies, power refers to the probability that an effect will be detected if it is truly there. A study that has sufficient power to detect an effect of a vaccine on incidence of disease will not necessarily have sufficient power to detect an effect on mortality. Deaths are more rare than disease and therefore larger sample sizes are needed to see a significant effect. 12.3: Methodological issues 1. Case definition and ascertainment

  • The type of ascertainment will determine the type and severity of disease. For example, individuals only attend health facilities when they feel quite sick, and cases admitted to hospital will be particularly severe. Interaction: Hyperlink: ascertainment: Ascertainment The method by which something is found out. In epidemiology this usually refers to the method of detecting cases. It is also important that equal effort is made to detect cases among vaccinated and unvaccinated populations. Which of the following is more likely to give a biased estimate of the protective effect of the vaccine in preventing disease?

    Interaction: Hotspot: Ascertainment at health facilities: Correct Response (pop up box appears): Thats correct. Ascertainment at health facilities may be biased towards those individuals that are more likely to access the health services, and are therefore more likely to get vaccinated. Active case detection of the population but making house-to-house visits is likely to give the least biased estimate of VE. Interaction: Hotspot: Active case detection in the community: Incorrect Response (pop up box appears): No, in fact active case detection of the population by making house-to-house visits is likely to give the least biased estimate of VE. Ascertainment at health facilities may be biased towards those individuals that are more likely to access the health services, and are therefore more likely to get vaccinated. 12.4: Methodological issues 2. Comparability between groups Whether you are comparing vaccinated and unvaccinated individuals, or cases and controls, it is important to control for confounding factors. The two groups should ideally be similar in as many ways as possible. 12.5: Methodological issues 2. Comparability between groups Some of the most common confounding factors in infectious disease epidemiology are age and socio-economic status. However, achieving such comparability is often difficult in observational studies, as vaccination uptake is not uniform in most societies.

    Ascertainment at health facilities

    Active case detection in the community

  • Some individuals are more likely to be vaccinated and also less likely to be infected. A prime example is BCG vaccination which has been conducted through schools, leaving-out the less fortunate individuals who do not attend school. In such circumstances it is important to collect data to control for confounding. 12.6: Methodological issues 2. Comparability between groups It is also important that the two groups have had a similar exposure to the infection of interest. This is easiest in settings where the incidence of disease is relatively high, such as in contact studies. The method of determining vaccination status should also be the same for cases and non-cases, and the method of ascertaining cases should be the same between vaccinated and unvaccinated individuals. Likewise, the determination of vaccination status should, as far as possible, be 'blind' to the illness status of the individual and vice-versa. 12.7: Methodological issues 3. Time Issues Individuals should not be included in VE estimates if they are younger than the age of recommended vaccination. In a matched case-control study, the vaccine status of the controls should be determined at the same age at which the case developed disease. For vaccines requiring more than one dose, efficacy can be calculated for the complete course, and after each dose. The comparison group should always be those that have received no doses. A similar approach can be used for vaccines that are given at different ages, again using the unvaccinated as the comparison group. 12.8: Methodological issues 3. Time Issues The efficacy of a vaccine can sometimes decline with time. To investigate this, VE can be estimated for different periods of time since vaccination. Many vaccines will only work if an individual is vaccinated before the infection is established.

    In addition, most vaccines take some time (at least 1 to 2 weeks) to induce a protective immune response. For this reason, most VE studies for diseases of short incubation will exclude cases that occur less than 2 weeks (sometimes plus the incubation period of the disease) after the last dose. 12.9: Methodological issues Time for reflection As this session is quite long, we suggest that this is a good time to take a break.

  • Read the papers from this session in full, looking closely at the methodological issues that have just been discussed. Think about whether each of these studies took these methodological issues into consideration, and how. Think about the benefits and deficits of each paper.

    Use this reading to help consolidate what you have just learned.

    After your break, you can continue with the remainder of the session and a few exercises. Section 13: Vaccination in practice Once vaccination is initiated at a population level, the estimates of vaccine efficacy will be different from those under trial conditions.

    This will be partly due to the ability of the programme to correctly identify the population at risk, achieve high vaccine uptake among that target population and maintain efficient vaccination practice. Once the programme is able to achieve high coverage of the target population, this will have an impact on the overall levels of transmission of the infection.

    These issues will be considered on this page. 13.1: Vaccination in practice Direct & indirect effects Immunisation of large numbers of people reduces the number of susceptible individuals. As this reduction nears the critical threshold of susceptible individuals (Review this in Session EC06), the transmission of the infection will decline. The probability that an unvaccinated susceptible individual will encounter an infectious individual and become infected will also decline. This effect is known as herd immunity, as each member is protected by the immune status of the total population ('herd') irrespective of their own immune status. 13.2: Vaccination in practice Direct & indirect effects Vaccination programmes can therefore work at two levels:

    1. the protection of the individual by inducing an effective immune response: direct, personal protection, and 2. the protection of unvaccinated individuals in the population by reducing the number of cases (and possibly the degree and duration of infectiousness) and therefore the amount of transmission of the infectious agent: indirect, herd immunity.

  • The total population impact of a vaccination programme will include both these direct and indirect effects of vaccination. 13.3: Vaccination in practice Direct & indirect effects In most situations, vaccinated and unvaccinated individuals interact, so the indirect effect will be the same among the vaccinated and the unvaccinated individuals. However, the method you have been using to calculate VE is based on the relative risk of disease in the vaccinated compared with the unvaccinated individuals.

    So, if there is a reduction in incidence in the whole population due to the indirect effects of vaccination, this will not be detected by the conventional methods. These methods only measure the direct effects. 13.4: Vaccination in practice Direct & indirect effects If vaccinated and unvaccinated individuals do not mix, the exposure to infection will be different between the two populations. In this case, both direct and indirect effects will only affect the vaccinated population. The relative risks will then reflect the combined effects of vaccination. To distinguish between these direct and indirect effects, different RCT designs have been suggested (Struchiner et al. 1990) involving some randomisation at the community level. As the effects are occurring at a population level, several populations or communities should be included in the RCT. Interaction: Hyperlink: community: Community For example, a village, school, or district. 13.5: Vaccination in practice Direct & indirect effects Interaction: Tabs: Populations: The diagram opposite shows how such a study would be structured, with each population representing a number of communities. Interaction: Tabs: RCT: The communities would be randomised to receive some vaccine (A) or no vaccine (B). Within the population A communities, the individuals would then be randomised to receive vaccine or no vaccine.

  • Different combinations of comparisons between these groups would then allow different measures of VE to be estimated. Interaction: Tabs: Exercise: Which comparison would estimate only the direct effects of vaccination?

    Interaction: Hotspot: Vaccinated in A versus unvaccinated in B: Incorrect Response (pop up box appears): In fact, the direct effects of vaccination can be estimated by comparing vaccinated and unvaccinated individuals in the same community (A), as both will be exposed to the same indirect effects. The effects estimated by the other comparisons will be considered on the next card. Interaction: Hotspot: Unvaccinated in A versus unvaccinated in B: Incorrect Response (pop up box appears): In fact, the direct effects of vaccination can be estimated by comparing vaccinated and unvaccinated individuals in the same community (A), as both will be exposed to the same indirect effects. The effects estimated by the other comparisons will be considered on the next card. Interaction: Hotspot: Vaccinated in A versus unvaccinated in A: Correct Response (pop up box appears): Thats right, a comparison of vaccinated and unvaccinated individuals in the same community (A) would provide an estimate of the direct effects of vaccination.

  • Interaction: Hyperlink: Source (pop up box appears): Source: After Halloran ME, Longini IM Jr. and Struchiner CJ. Design and interpretation of vaccine field studies. Epidemiological Review 1999; 21(1): 73-88. 13.6: Vaccination in practice The boxes below show the effects estimated by different comparisons between communities.

    Complete the diagram by dragging the correct effects into the boxes on the right. Interaction: Drag and Drop: Indirect effects: Correct Response middle box on RHS(pop up box appears): Thats correct. Transmission will have been reduced in A communities because of herd immunity effects. Therefore, a comparison of unvaccinated individuals in the vaccinated communities with unvaccinated individuals in the unvaccinated communities will measure the indirect effects of vaccination. Incorrect Response:

  • in fact, transmission will have been reduced in A communities because of herd immunity effects. Therefore, a comparison of unvaccinated individuals in the vaccinated communities with unvaccinated individuals in the unvaccinated communities will measure the indirect effects of vaccination. Interaction: Drag and Drop: Direct plus indirect effects: Correct Response bottom box on RHS (pop up box appears): Thats correct. Individuals in unvaccinated communities will not be subject to any effects of vaccination, while vaccinated individuals will be subject to both effects. So a comparison of vaccinated individuals in the vaccinated community with unvaccinated individuals in the unvaccinated community will estimated direct and indirect effects. Incorrect Response (pop up box appears): In fact, Individuals in unvaccinated communities will not be subject to any effects of vaccination, while vaccinated individuals will be subject to both effects. So a comparison of vaccinated individuals in the vaccinated community with unvaccinated individuals in the unvaccinated community will estimated direct and indirect effects. Interaction: Drag and Drop: Overall effects: Correct Response top box on RHS: Thats correct. A comparison of all the individuals in the vaccinated communities with all the individuals in the unvaccinated communities will measure the overall effects of vaccination. This will include the direct and indirect effects on vaccinated individuals plus the indirect effects on unvaccinated individuals. Incorrect Response (pop up box appears): In fact, a comparison of all the individuals in the vaccinated communities with all the individuals in the unvaccinated communities will measure the overall effects of vaccination. This will include the direct and indirect effects on vaccinated individuals plus the indirect effects on unvaccinated individuals.

  • 13.7: Vaccination in practice Efficacy & effectiveness Routine vaccination programmes are unlikely to be able to maintain the ideal conditions of a phase 3 vaccine trial. There may be problems in the field relating to vaccine delivery, cold chain conditions, administration of correct dosages, etc. Population coverage may not be optimal, with the populations most at risk being the least accessible to the vaccination programme and vice versa. In addition there will be some indirect effects of population-scale vaccination. For these reasons, a distinction has been made between vaccine efficacy, which is due to immunisation under optimal conditions, and vaccine effectiveness, which takes into account the practical and programmatic conditions.

    With the exception of the RCT, the methods described here generally assess the effectiveness of the vaccine once it has become a programmatic activity. However there have also been suggestions that vaccines be tested under programmatic conditions in pragmatic effectiveness trials prior to licensing.

  • 13.8: Vaccination in practice

    Other population-level effects In addition to the reduction in exposure to infection, mass vaccination programs will also have other effects on the profile of the infection.

    As you saw in session EC06, a reduction in the proportion of susceptible individuals below the critical threshold is responsible for long-term cycles of disease. As mass vaccination will further reduce the proportion of susceptible individuals below natural levels, it will take longer for numbers of susceptible individuals to accumulate to levels sufficient for transmission. This means that vaccination programmes will have the effect of lengthening the inter-epidemic period. 13.9: Vaccination in practice Other population-level effects In the same way, the age profile of the disease may be changed.

    The average age at which individuals first get infected is directly related to the transmissibility of the infection, and this can be mathematically described by the following formula:

    R0 = L / A

    where L is the average life-expectancy and A is the average age at first infection. Session 3 covers how this formula is derived and the assumptions made (see EC03p10 for details). As R0 decreases, the average age at first infection increases.

    Think about this intuitively. As the number of susceptible individuals declines and the number of infectious cases declines, the probability of making effective contact for the transmission of an infection will also decline. This means that a susceptible individual can progress through more of his or her life before encountering an infectious individual.

    Click the button below to see a graph of this. Interaction: Button: Graph:

  • This graph shows the changes observed in the age distribution of percentages of cases for three infections following the introduction of a mass vaccination programme in Bangkok (Nokes & Anderson 1988). Source Nokes DJ and Anderson RM. The use of mathematical models in the epidemiological study of infectious diseases and in the design of mass immunization programmes. Epidemiology and Infection 1988; 101(1): 1-20. 13.10: Vaccination in practice Other population-level effects As the average age at first infection increases, this may have practical implications for measurement of vaccine efficacy and for the outcome of the infection. It is important that the measurement of vaccine efficacy takes into account such possible shifts in age distribution.

    For example, if the disease is mainly a problem in under-5 year olds, an evaluation of routine vaccination programmes should also look for possible increases in incidence among older children. 13.11: Vaccination in practice Other population-level effects Although, in general, an upward age shift reduces the overall burden of disease and mortality, in some cases this shift may worsen the situation.

    A well-known example is rubella. After the introduction of routine rubella vaccination, the age profile shifted upwards so that women were more likely to become infected during pregnancy. In women in their first trimester of

  • pregnancy, infection can cause foetal abnormalities, miscarriage or stillbirth, known as congenital rubella syndrome (CRS). While national vaccination programmes reduced the incidence of rubella, they also caused an increase in the incidence of CRS.

    The solution to this was to introduce 'catch-up' vaccination programmes in schools for adolescent women, to ensure high vaccine uptake prior to their childbearing years. 13.12: Vaccination in practice Other population-level effects One final interesting feature is what occurs in relation to the age shift following the introduction of a mass vaccination programme.

    The first graph opposite shows the age profile of susceptible and immune individuals (hatched area) with a 'valley' of susceptibles that have lost maternal antibodies but have not yet acquired immunity from natural infection (S1). The second graph shows the situation after vaccination where some of the 'valley of susceptibles' has been filled by immunisation. Interaction: Tabs: Before immunisation:

    Interaction: Hyperlink: Source: Source: Nokes DJ and Anderson RM. The use of mathematical models in the epidemiological study of infectious diseases and in the design of mass immunization programmes. Epidemiology and Infection 1988; 101(1): 1-20. Interaction: Tabs: After immunisation:

  • Interaction: Hyperlink: Source (pop up box appears): Source: Nokes DJ and Anderson RM. The use of mathematical models in the epidemiological study of infectious diseases and in the design of mass immunization programmes. Epidemiology and Infection 1988; 101(1): 1-20 13.13: Vaccination in practice Other population-level effects (RHS remains static - graphs) The initial reduction in transmission will result in the total proportion of susceptible individuals remaining approximately equal to that prior to immunisation. However, the average age of the susceptibles has increased.

    This situation will change as successive cohorts of children are immunised and the ' valley ' gets filled in. After some years the proportion of susceptibles will decrease, although the average age of susceptibles will have increased (S2). Section 14: Mechanisms of protection An issue that has been much discussed in the literature is the epidemiological implication of the mechanism by which a vaccine works.

    Two simple mechanisms of vaccine protection have been proposed: 1. All or nothing This implies that the vaccine induces complete protection in some individuals, but has no effect on the remainder, leaving them susceptible. 2. Partial This implies that the vaccine induces some degree of immunity in all those vaccinated.

  • 14.1: Mechanisms of protection The diagrams opposite represent the two hypotheses. Interaction: Tabs: All or nothing: In the first scenario, a vaccine efficacy of 80% would correspond to 80% of vaccinated individuals becoming totally immune, and the other 20% remaining totally susceptible.

    This means that the number of individuals at risk is reduced by 80%, but the risk of disease remains the same in the 20% that are vaccinated but un-immunised. Interaction: Tabs: Partial: In the second scenario, all vaccinated individuals reduce their susceptibility by 80%. This means that the number of individuals at risk stays the same, but the risk of disease is reduced by 80% in all vaccines. (when LHS is on tab All or nothing)

    (when LHS is on tab Partial)

    14.2: Mechanisms of protection The mechanism of protective immunity has a subtle implication for the use of risks and rates when measuring vaccine efficacy.

    In situations where the disease is rare

  • (e.g. less than 20% of vaccinated individuals could get infected), risks and rates will be very similar.

    For more common diseases, the two measures of incidence will be different, and their usage will vary according to the mechanism of immune protection (Smith et al. 1984). 14.3: Mechanisms of protection In a theoretical situation where individuals can be followed up for an infinite time: Interaction: Tabs: Partial: For the mechanism of partial immunity, the proportion of vaccinated individuals that do not get infected will gradually decrease with time. This is because all individuals have a reduced risk, but are not fully protected (model 1, opposite).

    In this case, VE is better measured using incidence rates. VE using risks will decline with time, assuming all vaccinated individuals get infected if they are followed up for long enough. Interaction: Tabs: All or nothing: For the 'all or nothing' mechanism, the proportion of vaccinated individuals that do not get infected will decrease until all the non-immune vaccinees are infected and the remaining vaccinees are all immune (model 2 opposite).

    In this case, VE will be better measured using risks. VE using rates will increase to 100% with time, because eventually all the remaining vaccinees will be fully protected. (when LHS tab is on Partial)

    (when RHS tab is on All or nothing)

  • 14.4: Mechanisms of protection Using both measures (risks and rates), it may be possible to identify the mechanism of immune protection provided by a vaccine.

    If the VE with risks declines with time, but the VE with rates remains constant, it suggests a partial immunity mechanism.

    If the VE with risks remains constant but the VE with rates increases with time, it suggests an 'all or nothing' mechanism. In reality, many vaccines probably work by some hybrid mechanism, with different vaccinees protected to different degrees. This may be determined by their genetic make-up, prior immune status, and differences in delivery and administration of the vaccine. Section 15: Exercises The following exercises are designed to help you review the material you have covered in this session.

    Exercise 1 Consider the following scenarios, below and opposite: (a) A country decided to suspend vaccination against pertussis because vaccine coverage among cases was the same as in the population as a whole. Think about the reasons for this

    decision do you agree with it? Interaction: Button: clouds picture (pop up box appears): Substituting the values into the formula

  • VE = (PPV PCV) / (PPV[1 PCV]) x 100, you can see that the decision was correct. If PCV = PPV, then the numerator will be 0 and VE will be equal to 0. This situation occurred in Sweden in 1980, and led to Sweden being the only country in the world not to give pertussis vaccine routinely to children.

    (b) 80% of children are vaccinated against measles in your region. You are getting many calls from the primary health care workers, worried about the efficacy of measles vaccine as half the cases are vaccinated. Do you have a problem? Write down

    your reasons. Interaction: Button: clouds picture (pop up box appears): Substituting the proportion of cases and of the overall population that was vaccinated into the formula: VE = (PPV PCV)/(PPV[1 PCV]) x 100, you get: VE = (0.8 - 0.5)/(0.8 x [1 0.5]) x 100 = 75%. This implies that the measles vaccine is 75% effective in your region. If this is correct, then you would expect 50% of the cases to be vaccinated if the overall vaccine uptake is 80%. One could argue that a 75% VE for measles vaccine, is still very low, considering how infectious measles is. The basic reproductive number for measles is 12 to 18, implying that at least 90% of the population needs to be immune to stop transmission. 15.1: Exercises Remember, when you compare vaccine coverage in cases and in the population in this simple way, there is no control for confounding factors. You don't know whether the cases are really representative of the whole population.

    The case-population method is a 'quick and dirty' method for initial evaluation, and helps to determine whether there is likely to be a cause for concern. However, the result could be misleading, so it needs to be considered critically by someone with a sense of the population concerned. 15.2: Exercises Exercise 2

    A double-blind RCT of a Hepatitis B virus (HBV) vaccine was carried-out among homosexual men attending sexually transmitted disease clinics.

    Three doses of vaccine were given: at entry, after one month and after six months. Serological testing was done at 2, 4 and 8 months after entry into the study. The main outcome of interest was the presence of HBV surface (HBs) antigen in blood, which is a marker of HBV infection. The incubation period for HBV ranges from 6 weeks to 6 months.

  • The results of the trial are summarised below:

    Calculate the vaccine efficacy in each case, and then answer the questions on the next cards. 15.3: Exercises Interaction: Tabs: 1: In the study, the investigators excluded the first 3 months of follow-up time. Why do you think they did this?

    Hint: see http://www.cdc.gov/hepatitis/HBV/HBVfaq.htm#C3 Interaction: Button: clouds picture (pop up box appears and text appears on bottom LHS): It may take 3 months from infection to the appearance of HBs antigen the actual range of time for seroconversion is 2 weeks to 6-9 months (in rare cases). The first 3 months were excluded because the vaccine might not be expected to protect against infection acquired before first vaccination. Was this decision justified by the data? Interaction: Button: clouds picture (pop up box appears): The data suggest that this decision was justified. The VE in the first 3 months was low at 18% [VE = (16.6 13.6)/16.6 x 100 = 18.1%], while after 3 months it was much higher at 82% [VE = (13.5 2.5)/13.5 x 100 = 81.5%]. The overall efficacy was

  • VE = (14.3 4.9)/14.3 x 100 = 65.7%, so by including infections that may have arisen before vaccination, the overall VE is underestimated. Interaction: Tabs: 2: Compare the rate of infection among those receiving placebo during the first 3 months with that after 3 months. How might this be explained? Interaction: Button: clouds picture (pop up box appears and text appears on bottom LHS): The rate of infection in the placebo group is higher in the first 3 months compared with afterwards. This may be because there has been a change in risk behaviour associated with participating in the trial (perhaps as a reaction to the information given at enrolment). Other reasons may include a long-term increasing trend in overall population rates of infection, contamination of the placebo group (if some controls were in fact vaccinated), or perhaps a saturation effect where those at greater behavioural risk became infected early in the study leaving only less susceptible individuals. The decline in risk is unlikely to have been due to the impact of vaccination on transmission (indirect effect), as this was a study and only a small proportion of the population at risk in each city was involved. There would have been a sufficient reservoir of infection remaining among the non-study participants. 15.4: Exercises The table below shows the incidence of HBs antigens according to the level of anti-HBs antibodies after vaccination. Calculate the VE to the nearest percent for each level of seroconversion and fill in the missing cells.

    Interaction: Calculation: VE/Vaccinated(low): Correct Response 5 (pop up box appears): Correct Yes, the vaccine efficacy is given by: VE = 100 x (13.45 - 12.77)/13.45 = 5%

  • Incorrect Response (pop up box appears): No, remember that the vaccine efficacy is given by: VE = 100 x (Iu Iv) / Iu VE = 100 x (13.45 12.77)/13.45 = 5% Interaction: Calculation: VE/Vaccinated(med): Correct Response 14 (pop up box appears); Correct Yes, the vaccine efficacy is given by: VE = 100 x (13.45 11.57)/13.45 = 14% Incorrect Response (pop up box appears): No, remember that the vaccine efficacy is given by: VE = 100 x (Iu Iv)/Iu VE = 100 x (13.45 11.57)/13.45 = 14% Interaction: Calculation: VE/Vaccinated(high): Correct Response 98 (pop up box appears): Correct Yes, the vaccine efficacy is given by: VE = 100 x (13.45 0.28)/13.45 = 98% Incorrect Response (pop up box appears): No, remember that the vaccine efficacy is given by: VE = 100 x (Iu Iv)/Iu VE = 100 x (13.45 0.28)/13.45 = 98% Based on your estimates of VE, which mechanism of vaccine protection (partial, or 'all or nothing') do you think best fits this vaccine?

    Write down your reasons. Interaction: Button: clouds picture (pop up box appears): Individuals with low antibody levels had almost no protection and those with intermediate antibody levels had little protection from the vaccine. Individuals with high antibody levels had almost complete protection. An initial look at these results suggests that the vaccine might work through an all or nothing mechanism. To assess this properly the results should be stratified by more antibody categories to see if there as a sharp rise in protective effect at some particular threshold level. 15.5: Exercises Exercise 3

  • You are planning a retrospective study of measles vaccine efficacy. To avoid misdiagnoses you decide that all cases of measles must be confirmed by serology.

    Will that satisfy your concerns about the effect of disease misclassification on VE estimates?

    Incorrect Response Yes (pop up box appears): In fact it will not. As the study is retrospective, it will not be possible to tell whether the seropositive status resulted from measles infection or from vaccination. Correct Response No (pop up box appears): Thats right. As the study is retrospective, it will not be possible to tell whether the seropositive status resulted from measles infection or from vaccination. 15.6: Exercises Exercise 4

    A RCT was conducted to estimate the efficacy of a vaccine against meningococcal B disease. 700 schools were randomised to receive the vaccine and 660 to receive a placebo. During the follow-up 26 cases were identified in 26 'placebo' schools and 14 cases in 13 'vaccinated' schools. Calculate the vaccine efficacy, remembering to use the risk of having any infection in a school. Interaction: Button: clouds picture (pop up box appears and card appears on right handside): Vaccine efficacy was estimated as: VE = 1 [(13/700) / (26/660)] = 57.1% This is because the unit of randomisation was the school, so school outbreaks rather than individual cases were counted. A more complex analysis could have considered the number of cases and the number of person-years at risk in each school. Interaction: Tabs: Question i: Which of the following does this VE measure?

    Indirect

    Direct

    Direct and indirect

    Overall

  • Interaction: Hotspot: Direct: Incorrect Response (pop up box appears): Thats not right. All the students in each school received either the vaccine or placebo. Assuming that children from the different schools do not interact and most transmission occurs within school, children in vaccinated schools would benefit from both direct and indirect effects of vaccination, while those in placebo schools receive neither. Please try again. Interaction: Hotspot: Indirect: Incorrect Response (pop up box appears): Thats not right. All the students in each school received either the vaccine or placebo. Assuming that children from the different schools do not interact and most transmission occurs within school, children in vaccinated schools would benefit from both direct and indirect effects of vaccination, while those in placebo schools receive neither. Please try again. Interaction: Hotspot: Direct and indirect: Correct Response (pop up box appears): Correct. All the students in each school received either the vaccine or placebo. Assuming that children from the different schools do not interact and most transmission occurs within school, children in vaccinated schools would benefit from both direct and indirect effects of vaccination, while those in placebo schools receive neither. Interaction: Hotspot: Overall: Incorrect Response (pop up box appears): Thats not right. All the students in each school received either the vaccine or placebo. Assuming that children from the different schools do not interact and most transmission occurs within school, children in vaccinated schools would benefit from both direct and indirect effects of vaccination, while those in placebo schools receive neither. Please try again. Interaction: Tabs: Question ii: What are the advantages and disadvantages of this study design over a RCT randomised at the individual level? Interaction: Button: clouds picture (pop up box appears): The advantage is that both direct and indirect effects of vaccination can be measured. Also it is logistically more simple to allocate the same treatment to a whole school as there are no problems about treatments getting mixed-up. However, it requires a much larger sample size to control for confounding effects as the unit of randomisation is the group, which is itself large. 15.7: Exercises

  • Exercise 5

    The data in the table opposite are from a household contact study of pertussis vaccine efficacy. The secondary attack rates (SARs) and vaccine efficacy (VE) have been calculated for each age group.

    Is 65% a good estimate of overall vaccine efficacy? Interaction: Button: clouds picture (pop up box appears): The VE calculated overall is higher than any of the age-specific VE estimates. It is important to calculate a weighted-average summary estimate (e.g. using the Mantel-Haenszel method). In addition, age is a confounding factor as the unvaccinated young children have higher SARs, and also a lower vaccine coverage. This will often be the case in vaccine studies, so it is important to control for confounding factors!

    Section 16: Summary This is the end of EC07. When you are happy with the material covered here please move on to session EC08. The main points of this session will appear below as you click on the relevant title. Vaccine efficacy Interaction: Hyperlink: Vaccine efficacy:

  • Vaccine efficacy is the percent reduction in incidence (risk, rate, etc.) among vaccinated (not necessarily immunised) individuals, which is attributable to vaccination.

    It is calculated using the following formulae, which are equivalent to each other:

    Vaccine efficacy (VE) = Iu Iv x 100

    Iu where:

    Iu = incidence in unvaccinated individuals Iv = incidence in vaccinated individuals

    VE = (1 Relative Risk) x 100_ Methods to estimate vaccine efficacy 1

    The following seven methods can be used to estimate vaccine efficacy. 1) Randomised controlled trials (RCTs): Considered to be the ideal method because they control for confounding and biases. They are generally used prior to licensing a vaccine and measure vaccine efficacy under trial

    conditions. Methods to estimate vaccine efficacy 2_

    2) Cohort studies: Not commonly used and have problems of confounding because of non-random allocation of vaccines. 3) Outbreak investigations: Can be very useful because there is usually a high incidence of disease and homogenous exposure to infection during an outbreak. However, an outbreak may be due to a local problem (such as a cluster of vaccination failures), in which case the results should not be

    generalised.

    Methods to estimate vaccine efficacy 3_

    4) Contact studies: Especially used for infections that are efficiently transmitted within a household, and have the advantage of homogeneous exposure to infection. 5) Retrospective cross-sectional studies: Used by EPI to assess the effectiveness of vaccines in routine programmes. The major drawback is

  • the low specificity of diagnosis. Methods to estimate vaccine efficacy 4

    6) Case-control studies: A common method of assessing VE as overall numbers can be small while still ascertaining sufficient cases. Choice of cases and controls need to be representative to avoid confounding and bias. 7) Case-population studies: A crude method used to rapidly assess vaccines in routine programmes. This relies on comparing the proportion vaccinated among cases with that among the total population:

    VE = (PPV PCV) / (PPV x [1 PCV])

    Important methodological issues 1_ Important issues are:

    1) Case definition This needs to be as

    specific as possible, especially as vaccines may confer different levels of protection against different forms of

    clinical disease.

    2) Ascertainment The type of ascertainment will determine the type and severity of disease, and also the distribution of vaccinated or unvaccinated individuals.

    Important methodological issues 2_ 3) Comparability To prevent

    confounding it is necessary that cases and controls are compara