Epidemiology Workshop II Oak Ridge Reservation Health Effects Subcommittee December 3, 2001...

Epidemiology Workshop IIOak Ridge Reservation

Health Effects SubcommitteeDecember 3, 2001

Evaluating an Epidemiologic Study

Lucy Peipins, PhD

Sherri Berger, MSPH

Division of Health Studies

How Science Works

Every epidemiologic study and every analysis is built on a body of knowledge and is done to expand that body of knowledge

Is there a relationship between a particular exposure and a disease?


• What is the purpose of scientific publication?

– To present a new finding. Results are shared and debated with other scientists and the public

– To demonstrate validity of study results by presenting the study methods, design, analysis and interpretation.

– This provides the opportunity for others to judge the quality of the study


• Like all scientists, epidemiologists have criteria to judge the scientific quality and validity of a study

– Has the author presented a good case for his or her hypothesis?


• Each journal judges the scientific content and validity of an article by a peer review process

• Reviewers typically do not know the identity of the author

• Reviewers make recommendations on whether the article should be published


• To illustrate these criteria, we will use the article ‘Cancer Mortality near Oak Ridge, Tennessee’ by J. Mangano published in the International Journal of Health Services

• The criteria we outline can be applied to any study.


• The first thing we need to find out is…

Why was this study done?

or, another way of saying this is…

What hypothesis is the author testing?


• The main purpose was to examine change in cancer mortality in 94 counties within 100 miles surrounding Oak Ridge, Tennessee, in 1950-1952 and in 1987-1989, and compare this with mortality rate change in the U.S. and the Southeast

– Exposure of interest is radiation from Oak Ridge weapons production which began in the early 1940s


Specific Hypotheses:– Increase in all cancer mortality ‘near’ Oak Ridge should

exceed the national and regional areas– Within Oak Ridge area, increase in cancer mortality rate

should be greatest in rural areas– Increase in cancer mortality should be greatest ‘near’ the

weapons plant– Cancer mortality should be greatest in mountainous areas– Cancer mortality increases should be greater downwind of

weapons plant


• What type of study was this?

Descriptive study. This type of study describes health outcomes alone. It answers the “who, what, when and where” of a disease in a group of people.


Analytic study. This type of study answers “why” a population or group of people get a particular disease.


• This is primarily descriptive. Exposure is based on county of residence Hypotheses based on “where”-- counties nearest to weapons plant, counties downwind, mountainous counties, rural counties

It is also analytic because the comparison groups--Southeast or U.S.


• Keeping in mind that epidemiology is the study of groups of people or populations, who was the study group?


• White population living less than 100 miles (in 94 counties) from Oak Ridge between 1950-1952 and 1987-1989

– Comparison population was the white population of the Southeast and the U.S. in those time periods


• What was the exposure of interest?


• Several types of exposure were mentioned:

– radiation, or chronic, low-level ingested radiation,

– emissions from nuclear plant into both air and water, and radiation from consumption of milk and vegetables were also mentioned


• How was exposure defined and measured?


• Exposure was not measured. • Exposure was defined as residence in a

county near weapons plant, residence in mountainous counties, residence in downwind counties and residence in rural counties.


• Let’s review the hierarchy of exposure assessment from our first workshop and see how to characterize Mangano’s exposure classification

Measuring Exposure• We can measure exposures or their surrogates along the entire

path from emissions to body burden

Types of Data Approximation to Actual Exposure_________________________________________________________________________1) Residence in a defined geographical Poorest xarea (e.g. county) of a site |2) Residence in a geographical area in | xproximity to to a site where exposure is assumed | x3) Distance or duration of residence |4) Distance and duration of residence |5) Quantified surrogate of exposure (e.g. estimate |of drinking-water use |6) Quantified area measurements in vicinity of the |residence |7) Quantified personal measurement Best

Baker D, et al. Environmental Epidemiology: A Textbook on Study Methods and Public Health

Applications, 1999


• What are some limitations or criticisms you might have about how exposure was measured?


• Residence in county is a poor measure of exposure.

– All persons living in the 94 counties are assigned the same exposure

– Persons who lived in the county at the time of death may not have been long-term residents or persons who were long-term residents may have retired, moved and died in another county or state

– All persons in the rest of the southeast or U.S. are assigned no exposure


• Proximity to site is poorly defined in this study.

– Why is Roane county not considered to be near the site?

– Why are the comparison 12 counties within 40 miles of the site and not further away?

– Different pathways of exposure (water and air pathways) may

exert a stronger influence than vicinity alone


• Mountainous vs. non-mountainous counties

– Rainfall in mountain areas is higher that valleys in general, but there is a great deal of variation between mountainous counties and between mountains and valleys in each county. Category of ‘mountainous’ is too general.


• No rationale (or references) for the assumption of urban/rural differences in cancer mortality.

– Why select Georgia, NC, SC and Tennessee and not include Kentucky and Virginia?

– No references on why rural population might be a lower risk of cancer than urban populations.

– Rural areas are not free of risks -- pesticides and herbicides, smoking, and others


• Downwind counties

– We have more information from dose reconstructions that contradicts some of the hypothesized wind patterns


• What was the outcome of interest?

– Deaths from cancer among whites in 94 counties between 1950-1952 and 1987-1989 compared with cancer deaths in U.S. and Southeast.


• How was the health outcome measured in this study?

– Cancer mortality data by counties were obtained from the National Cancer Institute

– Based upon death certificates


• What are some limitations or criticisms you might have about how the outcome was chosen and measured?


• All cancers together rather than specific cancers were considered.

– Remember, cancer is not one disease but over 100 different diseases arising from 40 sites in the body

– There are many causes of these cancers-not all can be linked to environmental causes.


• There is considerable inaccuracy in death certificate data (25% error)– Accuracy may vary by region-depending on who

codes the certificates

– Accuracy may vary by time. Improvements in diagnosis or coding may have led to an increase in cancer rates over time


• Selection bias - could there have been bias (distortion or error) in how study subjects got selected into the study?

– Mangano used already existing data so subjects were not selected.


• Information bias - could there have been bias (distortion or error) in how information was collected from the study group and the comparison group or groups?

– Accuracy of cause of death certification may vary by region of the country


• Could the results have been due to confounding?

• Confounding is the possibility that the relationship we see between exposure and disease could have been due, totally or in part, to other factors or differences between the study group and comparison group.


• Potential Confounders– Smoking, diet, physical activity, and occupational

exposures might have differed between the study counties and the rest of the Southeast or U.S.

– Author assumes little or no migration. Yet we know there was a large influx of people into the Oak Ridge area in the 1940s and a subsequent declined in population. Younger people have a lower risk of cancer.


– Life expectancy may differ by region over time. As life expectancy increased in the southeast, deaths from cancer might increase since it is primarily a disease of older people

– Other sources of environmental contamination may have varied by region

– Potential increase in pollution from utility and other plants in the area over time


• Did the author measure any confounders? No.

– No information on confounders is found on death certificates


• Describe the methods used to measure the relationship between exposure and outcome.

– Age-adjusted mortality rates (#deaths/population) for selected counties were calculated for 1950-1952 and 1987-1989 and compared to Southeast counties and U.S.


• Statistical Analysis– Very little information on statistical methods

including why 1950 standard was used – Why choose only 1950-52 and 1987-89? Why not

rates by decade or calculate average annual increases?

– Why not calculate rates for different age groups– Why not calculate rates for different cancers


• What are the major results of the study. Author states that: – The cancer mortality rate increase in the Oak

Ridge area (94 counties) exceeded the national and regional increases (30.5% vs. 5.1%)

– Rural areas experienced the greatest increase


– Counties downwind and in those in N.W. wind direction had greatest increase

– Anderson county had greater increase than selected counties < 40 miles from plant

– Most (but not all) mountainous areas had greatest increase


• Results were positive for all hypotheses, however,

– Has the author justified his conclusions?


• How strong was the association between exposure and outcome?– Large differences were seen between

study counties and the U.S.

– Not so strong for Southeast


• Has this relationship been seen by other researchers conducting other types of studies in different populations?

– Several studies have looked at residence near nuclear facilities. Worker studies have also looked at radiation and cancer.


• Is there a biologically plausible explanation between exposure and disease? Yes.

– Ionizing radiation is known to cause cancer. However, radiation levels could not be measured --county of residence was the surrogate for ionizing radiation


• Does the risk of disease increase with increasing exposure (dose-response).

– Not demonstrated. No doses calculated, therefore no dose-response


• Has exposure occurred before disease?

– Unknown for an unknown number of people due to moving into and out of study counties


• Authors conclusion:

– Analysis of change in cancer mortality suggests a link between radiation contamination and increased risk to populations living near nuclear sites


• Has the author convinced us of his conclusions?

• Did this study advance our knowledge about the relationship between exposure and disease?


• Limitations of study– No exposure measure – Limitations in use of

residence in county surrogate for radiation exposure– Limitations to use of death certificate data and

mortality rather than incidence of cancer– Lack of control for potential confounders

– Focus on all cancers rather than those most likely to be related to exposure


– Because of these limitations, this study does not effectively demonstrate a relationship between cancer mortality rates and radiation exposure


• Epidemiology is about measurement—A study is only as good as its measures of exposure and outcome—both are essential as are minimizing bias and confounding

Epidemiology Workshop II Oak Ridge Reservation Health Effects Subcommittee December 3, 2001...

Documents

Transcript of Epidemiology Workshop II Oak Ridge Reservation Health Effects Subcommittee December 3, 2001...