Define the Case List Candidate Causes Evaluate Data from the Case Evaluate Data from Elsewhere...

Define the Case

List Candidate Causes

Evaluate Data from the Case

Evaluate Data from Elsewhere

Identify Probable Cause

Detect or Suspect Biological Impairment

As Necessary: Acquire Data,

and Iterate Process

Identify and Apportion Sources

Management Action: Eliminate or Control Sources, Monitor Results

Biological Condition Restored or Protected

Decision-maker and

Stakeholder Involvement

Stressor Identification

Hip Pocket Guide to SI

CADDIS 2007

www.epa.gov/caddis

1

Table of Contents

page

List of Types of Evidence 2

Introduction 3


4


16

Multiple Evidence as a Form of Evidence

23

Scoring Evidence 26

Cross Walk of Types of Evidence

33

2

Types of Evidencepage

Types of Evidence that Use Data from the Case

Spatial/Temporal Co-occurrence with positive reference sites

5

Spatial/Temporal Co-occurrence through Time 6

Spatial/Temporal Co-occurrence Upstream Downstream Comparison

7

Temporal Sequence 8

Stressor-Response Relationship in the Field 9

Causal Pathway 10

Evidence of Exposure or Biological Mechanism 11

Manipulation of Exposure 12

Laboratory Tests of Site Media 13

Verified Predictions 14

Symptoms 15

Types of Evidence that Use Data from Elsewhere

Mechanistically Plausible Cause 17

Stressor-Response from Other Field Studies 18

Stressor-Response from Laboratory Studies 19

Stressor-Response from Ecological Simulation Models 20

Manipulation of Exposure at Other Sites 21

Analogous Stressors 22

Evaluating Multiples Types of Evidence as a Form of Evidence

Consistency of Evidence 24

Reasonable Explanation of the Evidence 25

Introduction

U.S. EPA's Stressor Identification (SI) Guidance Document (2000) describes a formal and rigorous process to identify stressors causing biological impairments in aquatic ecosystems, and a structure for organizing the scientific evidence supporting the conclusions. The SI process has been updated on the website www.epa.gov/caddis.

The Stressor Identification process consists of five steps depicted within the yellow box on the cover of this guide. The hip pocket guide is intended as a convenient source of brief descriptions, illustrations, and scoring of the types of evidence used to evaluate candidate causes of biological condition. For more detail please consult the CADDIS website.

3 4


The most powerful evidence is based on cause-effect relationships developed from data obtained from the place where the impairment occurs.

-Use data from the case to develop evidence that allows you to confidently eliminate very improbable causes.-Use symptoms to refute or diagnose a cause. -Use data from the case to build a body of evidence for those candidate causes that cannot be refuted or diagnosed. This evidence is used to identify the most probable causes. The more types of evidence and the more characteristics that support a candidate cause, the more likely that it IS the true cause. The more types of evidence that weaken the case for a candidate cause the more likely that it IS NOT the true cause.

Refuted causes require no further analysis, but are revisited when summarizing the overall conclusions.

Confirmed diagnoses by definition are probable causes and require no further analysis, but are also revisited when describing probable causes in the final summary.

All other candidate causes are further evaluated by comparison among remaining candidate causes after also developing evidence using data from other studies or locations.

http://www.epa.gov/ost/biocriteria/stressors/stressorid.html

http://www.epa.gov/ost/biocriteria/stressors/stressorid.html

http://www.epa.gov/caddis

5

EVIDENCE FROM THE CASE

Spatial/Temporal Co-Occurrence with positive reference sites

Supports

The biological effect must be observed where and when the cause is observed, and must not be observed where and when the cause is absent.

Spatial/Temporal Co-occurrence with Positive Reference Sites, RefutesThe impairment (dead fish) does NOT occur in similar ecosystems exposed to the same causal agent (effluent from the same industry) at another site.

Spatial/Temporal Co-occurrence with Positive Reference Sites, SupportsThe impairment (dead fish) occurs in similar ecosystems exposed to the same causal agent (effluent from the same industry) at another site

6

Supports

Refutes

Spatial/Temporal Co-Occurrence Through Time


Spatial/Temporal Co-occurrence Through Time at a Site, RefutesThe impairment (dead fish) occurs whether the causal agent occurs (the effluent is being discharged) or not (no discharge).

Spatial/Temporal Co-occurrence Through Time at a Site, SupportsThe impairment (dead fish) occurs when the causal agent occurs (the effluent is being discharged) but not when it does not occur (no discharge).

7

Supports

Refutes

Spatial/Temporal Co-Occurrence Upstream Downstream Comparison


Spatial/Temporal Co-occurrence with Upstream/Downstream Comparisons, RefutesThe impairment (dead fish) occurs both upstream and downstream of the source of the causal agent (effluent).

Spatial/Temporal Co-occurrence with Upstream/Downstream Comparisons, SupportsThe impairment (dead fish) occurs downstream of the source of the causal agent (effluent) but not upstream.

8

Supports

Refutes

Temporal SequenceThe cause must precede the biological effect.

Temporal Sequence, RefutesThe impairment (dead fish) occurred before the candidate causal agent (the effluent).

Temporal Sequence, SupportsThe impairment (dead fish) occurred only after initiation of the candidate causal agent (the effluent).

Stressor-Response Relationships from the Field, SupportsThe impairment (dead fish) is greatest where the causal agent (the effluent) is most concentrated, and the impairment diminishes (more live fish) as the agent is diluted.

Stressor-Response Relationships from the FieldAs exposure to the cause increases, intensity or frequency of the biological effect increases; as exposure to the cause decreases, intensity or frequency of the biological effect decreases.

9

Weakens

Supports

Stressor-Response Relationships from the Field, RefutesThe impairment (dead fish) increases as the causal agent (the effluent) is diluted, and the impairment is absent where the agent is most concentrated as well as upstream.

Supports

Refutes

Causal PathwaySteps in the pathways linking sources to the cause can serve as supplementary or surrogate indicators that the cause and the biological effect are likely to have co-occurred.

10

Causal Pathway, SupportsThe impairment (dead fish) occurs downstream of the source of the precursor (phosphate) and high algal production is sufficient to cause high early morning respiration resulting in observed low dissolved oxygen (the causal agent). Turbulence, which might have aerated the water, is absent.

Causal Pathway, RefutesThe impairment (dead fish) occurs downstream of the source of a precursor (phosphate) of the candidate causal agent (dissolved oxygen) but the proposed causal pathway is interrupted by turbulence which aerates the water, and the lack of periphyton on the cobble substrate confirms that there is no eutrophication, so another step is absent.

Evidence of Exposure or Biological Mechanism

Supports

Weakens

Measurements of the biota show that relevant exposure to the cause has occurred, or that other biological mechanisms linking the cause to the effect have occurred. Measurements of the biota show that relevant exposure to the cause has occurred, or that other biological mechanisms linking the cause to the effect have occurred.

11

Evidence of Exposure or Biological MechanismThe candidate cause (low dissolved oxygen) of the impairment (dead fish) causes surviving fish to swim at the surface and gulp. Observations of that behavior strengthen the candidate cause and observations that surviving fish are swimming below the surface weakens the candidate cause.

12

Manipulation of Exposure

Refutes

Field experiments or management actions that increase or decrease exposure to a cause must increase or decrease the biological effect.

Manipulation of Exposure, RefutesThe impairment (dead fish) occurs when the causal agent is present (the effluent) and when it is removed (the effluent is shut down).

Manipulation of Exposure, SupportsThe impairment (dead fish) occurs when the causal agent (the effluent) is present but not when it is removed (the effluent is shut down).

Supports

13

Supports

Weakens

Laboratory Tests of Site Media

Controlled exposure in laboratory tests to causes (usually toxic substances) present in site media should induce biological effects consistent with the effects observed in the field.

Laboratory Tests of Site Media, SupportsThe impairment (dead fish) occurs in water that is toxic (dead fish in beaker of effluent diluted in site water). Water from an unimpaired reference site with the same industry is nontoxic.

Laboratory Tests of Site Media, RefutesThe impairment (dead fish) occurs in water that is not toxic (live fish in beaker of effluent diluted in site water). Water from an unimpaired reference site with the same industry is also nontoxic.

14

Verified Predictions

Supports

Weakens

Knowledge of a cause's mode of action permits prediction and subsequent confirmation of previously unobserved effects.

Verified PredictionsA clever assessor predicts that if the impairment (dead fish) is due to a cholinesterase inhibiting pesticide, he would find that crustaceans would also be dead, but organisms without cholinergic systems would be alive. If he then searches for evidence at the site and finds dead crayfish and live rotifers (upper right panel), that supports the candidate cause. If the crayfish are alive (lower right panel), that weakens the candidate cause.

Diagnose or

Supports

Weakens

16

Symptoms

Biological measurements (often at lower levels of biological organization than the effect) can be characteristic of one or a few specific causes.

15

SymptomsThe impairment (low fish production) may be caused by many agents, but only a few cause the specific symptom (spinal deformity). Observation of that symptom supports candidate causes that produce that symptom (e.g., selenium) and weakens others.


Most candidate causes cannot be diagnosed or eliminated. Further evaluation is required that uses knowledge gained from laboratory studies and from past experiences and observations in other waterbodies.

For example, one of the most useful types of evidence from elsewhere uses the stressor-response relationships developed from laboratory studies. Familiar examples are single chemical, single species toxicity tests.

-Use data from elsewhere to weaken or support a particular candidate cause.

17 18

EVIDENCE FROM ELSEWHERE

Mechanistically Plausible Cause

Weakens

The relationship between the cause and biological effect must be consistent with known principles of biology, chemistry and physics, as well as properties of the affected organisms and the receiving environment.

Mechanistically Plausible CauseIf the impairment is a fish kill with dermal lesions, the candidate cause copper toxicity is not mechanistically plausible, because copper does not cause dermal lesions.

Stressor-Response Relationships from Other Field Studies

At the impaired sites, the cause must be at levels sufficient to cause similar biological effects in other field studies.

Supports

Weakens

Stressor-Response Relationships from Other Field StudiesField studies from the region that associate measurements of the candidate cause (orange dots) with the intensity of the effect (proportion of dead fish) can be used to generate an exposure-response model. If concentrations in site water at the time of the fish kill (the impairment) are in concentration range A, the candidate cause is weakened. If they are in range B it is strengthened.

19

Stressor-Response Relationships from Laboratory Studies

SupportsWeakens

Within the case, the cause must be at levels associated with related biological effects in laboratory studies.

Stressor-Response Relationships from Laboratory StudiesThe short-term lethality to fish of zinc (a candidate cause) has been tested in the laboratory and the results used to develop a concentration-response relationship. If concentrations in site water at the time of the fish kill (the impairment) are in concentration range A, the candidate cause is weakened. If they are in range B, it is strengthened.

20

Stressor-Response Relationships from Ecological Simulation Models

Supports

Within the case, the cause must be at levels associated with effects in mathematical models simulating ecological processes.

Stressor-Response Relationships from Ecological Simulation ModelsIf the impairment (dead planktivorous fish) is thought to be due to starvation rather than direct toxicity, a mathematical model can show how the loss of zooplankton is a function of exposure to the candidate cause (chloronaphthol) and the starvation of fish is a function of zooplankton abundance.

Manipulation of Exposure at Other Sites

At similarly impacted locations outside the case sites, field experiments or management actions that increase or decrease exposure to a cause must increase or decrease the biological effect.

21

Supports

Weakens

Manipulation of Exposure Elsewhere, WeakensThe impairment (dead fish) occurs when the causal agent is present (an effluent of the same type as at the impaired site) and when it is removed (the effluent is shut down).

Manipulation of Exposure Elsewhere, SupportsThe impairment (dead fish) occurs when the causal agent (an effluent of the same type as at the impaired site) is present but not when it is removed (the effluent is shut down).

Analogous Stressors

Supports

Agents similar to the causal agent at the impaired site should lead to similar effects at other sites.

22

Analogous StressorsIf evidence is lacking for the candidate cause (17B Tremblone) evidence for a similar agent (Tremblone) may be used to strengthen or weaken that candidate cause.

Consistency of Evidence

Supports

Weakens

Weakens

Confidence in the argument for or against a candidate cause is increased when many types of evidence consistently support or weaken it

24

Consistency of the EvidenceA candidate cause is strongly supported if all available types of evidence are consistently supportive. It is greatly weakened if all available types of evidence are consistently weakening. It is weakened if some types of evidence support and others weaken the candidate cause.

Evaluating Multiple Lines of Evidence as a Form of Evidence

There are two types of evidence that are not directly taken from the data. Rather they concern a types of evidence that evaluate how other types of evidence relate to one another.

Consistency evaluates if all the types of evidence support or weaken the argument for a cause.

Explanation of the Evidence evaluates if an inconsistent type of evidence is credible and if there is a reason to place less weight on that piece of evidence.

23

Explanation of the Evidence

Confidence in the argument for a candidate cause is increased when a post hoc mechanistic, conceptual, or mathematical model reasonably explains any inconsistent evidence.

24

Supports

Weakens

Reasonable Explanation of the EvidenceIf recovery has not occurred one month following elimination of the candidate cause (the effluent) then spatial/temporal co-occurrence is not complete. However, it may be explained by the fact that recolonization is blocked by a weir and enough time has not elapsed for recovery by reproduction, which might be observed a year later (upper panel). That explanation supports the candidate cause. There is no such explanation for the continuation of the impairment (more dead fish) one month after elimination of exposure to the candidate cause (lower panel). 25

Scoring Evidence

The score for the evidence from the case and from elsewhere is influenced by the quantity and quality of the evidence. The magnitude of the score is based on the likelihood of observing the effect due to the true cause rather than from a chance outcome. Therefore, the highest scores are given to the types of evidence that: • Use data from the case, • Are based on more than one piece of evidence, • Effectively link the causal agent with the effect.

All of the evidence is evaluated for consistency by looking at the overall pattern of scores. Confidence in the argument for or against a candidate cause is increased when it is supported by many types of evidence.R refutes

D diagnoses

+++ convincingly supports

- - - convincingly weakens

++ strongly supports

- - strongly weakens

+ somewhat supports

- somewhat weakens

0 neither supports nor weakens

NE no evidence

Types of Evidence that Use Data from the Case score

Spatial/ Temporal Co-occurrence

The effect occurs where or when the candidate cause occurs, OR the effect does not occur where or when the candidate cause does not occur.

+

It is uncertain whether the candidate cause and the effect co-occur.

0

The effect does not occur where or when the candidate cause occurs, OR the effect occurs where or when the candidate cause does not occur.

- - -

The effect does not occur where and when the candidate cause occurs, OR the effect occurs where or when the candidate cause does not occur, and the evidence is indisputable.

R

Temporal Sequence

The candidate cause occurred prior to the effect. +

The temporal relationship between the candidate cause and the effect is uncertain.

0

The candidate cause occurs after the effect. - - -

The candidate cause occurs after the effect, and the evidence is indisputable.

R

Stressor-Response Relationship in the Field

A strong effect gradient is observed relative to exposure to the candidate cause, at spatially linked sites, and the gradient is in the expected direction.

+ +

A weak effect gradient is observed relative to exposure to the candidate cause, at spatially linked sites, OR a strong effect gradient is observed relative to exposure to the candidate cause, at non-spatially linked sites, and the gradient is in the expected direction.

+

An uncertain effect gradient is observed relative to exposure to the candidate cause.

0

An inconsistent effect gradient is observed relative to exposure to the candidate cause, at spatially linked sites, OR a strong effect gradient is observed relative to exposure to the candidate cause, at non-spatially linked sites, but the gradient is not in the expected direction.

-

A strong effect gradient is observed relative to exposure to the candidate cause, at spatially linked sites, but the relationship is not in the expected direction.

- -

Types of Evidence that Use Data from the Case score

Causal Pathway

Data show that all steps in at least one causal pathway are present. + +

Data show that some steps in at least one causal pathway are present.

+

Data show that the presence of all steps in the causal pathway is uncertain.

0

Data show that there is at least one missing step in each causal pathway.

-

Data show, with a high degree of certainty, that there is at least one missing step in each causal pathway.

- - -


Data show that exposure or the biological mechanism is clear and consistently present. + +

Data show that exposure or the biological mechanism is weak or inconsistently present.

+

Data show that exposure or the biological mechanism is uncertain.

0

Data show that exposure or the biological mechanism is absent.

- -

Data show that exposure or the biological mechanism is absent, and the evidence is indisputable.

R

Manipulation of Exposure

The effect is eliminated or reduced when exposure to the candidate cause is eliminated or reduced, OR the effect starts or increases when exposure to the candidate cause starts or increases.

+ + +

Changes in the effect after manipulation of the candidate cause are ambiguous.

0

The effect is not eliminated or reduced when exposure to the candidate cause is eliminated or reduced, OR the effect does not start or increase when exposure to the candidate cause starts or increases.

- - -

The effect is not eliminated or reduced when exposure to the candidate cause is eliminated or reduced, OR the effect does not start or increase when exposure to the candidate cause starts or increases, and the evidence is indisputable.

R

Types of Evidence that Use Data from the Case Score


Laboratory tests with site media show clear biological effects that are closely related to the observed impairment. + + +

Laboratory tests with site media show ambiguous effects, OR clear effects that are not closely related to the observed impairment.

+

Laboratory tests with site media show uncertain effects. 0

Laboratory tests with site media show no toxic effects that can be related to the observed impairment.

-

Verified Predictions

Specific or multiple predictions of other effects of the candidate cause are confirmed. + + +

A general prediction of other effects of the candidate cause is confirmed.

+

It is unclear whether predictions of other effects of the candidate cause are confirmed.

0

A prediction of other effects of the candidate cause fails to be confirmed.

-

Multiple predictions of other effects of the candidate cause fail to be confirmed.

- - -

Specific predictions of other effects of the candidate cause fail to be confirmed, and the evidence is indisputable.

R

Symptoms

Symptoms or species occurrences observed at the site are diagnostic of the candidate cause. D

Symptoms or species occurrences observed at the site include some but not all of a diagnostic set, OR symptoms or species occurrences observed at the site characterize the candidate cause and a few others.

+

Symptoms or species occurrences observed at the site are ambiguous or occur with many causes.

0

Symptoms or species occurrences observed at the site are contrary to the candidate cause.

- - -

Symptoms or species occurrences observed at the site are indisputably contrary to the candidate cause.

R

Types of Evidence that Use Data from Elsewhere score


A plausible mechanism exists. +

No mechanism is known. 0

The candidate cause is mechanistically implausible. - -

Stressor-Response from Laboratory Studies

The observed relationship between exposure and effects in the case agrees quantitatively with stressor-response relationships in controlled laboratory experiments.

+ +

The observed relationship between exposure and effects in the case agrees qualitatively with stressor-response relationships in controlled laboratory experiments.

+

The agreement between the observed relationship between exposure and effects in the case and stressor-response relationships in controlled laboratory experiments is ambiguous.

0

The observed relationship between exposure and effects in the case does not agree with stressor-response relationships in controlled laboratory experiments.

-

The observed relationship between exposure and effects in the case does not even qualitatively agree with stressor-response relationships in controlled laboratory experiments, or the quantitative differences are very large.

- -

Stressor-Response from Other Field Studies

The stressor-response relationship in the case agrees quantitatively with stressor-response relationships from other field studies.

+ +

The stressor-response relationship in the case agrees qualitatively with stressor -response relationships from other field studies.

+

The agreement between the stressor-response relationship in the case and stressor-response relationships from other field studies is ambiguous.

0

The stressor-response relationship in the case does not agree with stressor-response relationships from other field studies.

-

There are large quantitative differences or clear qualitative differences between the stressor-response relationship in the case and the stressor-response relationships from other field studies.

- -29

Types of Evidence that Use Data from Elsewhere score

Stressor-Response Relationships from Ecological Simulation Models

The observed relationship between exposure and effects in the case agrees with the results of a simulation model.

+

The results of simulation modeling are ambiguous. 0

The observed relationship between exposure and effects in the case does not agree with the results of simulation modeling.

-


At other sites, the effect is consistently eliminated or reduced when exposure to the candidate cause is eliminated or reduced, OR the effect is consistently starts or increases when exposure to the candidate cause starts or increases.

+ + +

At other sites, the effect is eliminated or reduced at most sites when exposure to the candidate cause is eliminated or reduced, OR the effect starts or increases at most sites when exposure to the cause starts or increases.

+

Changes in the effect after manipulation of the candidate cause are ambiguous.

0

At other sites, the effect is not consistently eliminated or reduced when exposure to the cause is eliminated or reduced, OR the effect does not consistently start or increase when exposure to the cause starts or increases.

- -

Analogous Stressors

Many similar agents at other sites consistently cause effects similar to the impairment. + +

One or a few similar agents at other sites cause effects similar to the impairment.

+

One or a few similar agents at other sites do not cause effects similar to the impairment.

-

Many similar agents at other sites do not cause effects similar to the impairment.

- -

30

Evaluating Multiples Types of Evidence as a Form of Evidence Score

Consistency

All available types of evidence support the case for the candidate cause. + + +

All available types of evidence weaken the case for the candidate cause. - - -

All available types of evidence support the case for the candidate cause, but few types are available. +

All available types of evidence weaken the case for the candidate cause, but few types are available. -

The evidence is ambiguous or inadequate. 0

Some available types of evidence support and some weaken the case for the candidate cause. -

Reasonable Explanation of the Evidence

There is a credible explanation for any negative inconsistencies or ambiguities in an otherwise positive body of evidence that could make the body of evidence consistently supporting.

+ +

There is no explanation for the inconsistencies or ambiguities in the evidence. 0

There is a credible explanation for any positive inconsistencies or ambiguities in an otherwise negative body of evidence that could make the body of evidence consistently weakening.

-

31

Cross-walk of Types of EvidenceUpdated CADDIS (2006) Original SI Guidance

(2000)

Spatial/Temporal Co-occurrence

Co-occurrence & Consistency of Association from Site

Temporal Sequence Temporality

Stressor-Response Relationship in the Field

Biological Gradient

Causal Pathway Complete Exposure Pathway


Complete Exposure Pathway

Manipulation of Exposure Experiment


Experiment

Verified Predictions Predictive Performance

Symptoms Diagnosis & Specificity of Cause


Plausible Mechanism

Stressor-Response from Laboratory Studies

Plausible Stressor-Response & Consistency of Association

Stressor-Response from Other Field Studies

Plausible Stressor-Response

Stressor-Response from Ecological Simulation Models

Plausible Stressor-Response


Experiment

Analogous Stressors Analogy

Consistency of Evidence Consistency

Reasonable Explanation of the Evidence

Coherence of Evidence

32

CADDIS 2007

www.epa.gov/caddis

Define the Case List Candidate Causes Evaluate Data from the Case Evaluate Data from Elsewhere...

Documents

Transcript of Define the Case List Candidate Causes Evaluate Data from the Case Evaluate Data from Elsewhere...