1

This paper has been prepared for the Harvard Center for Risk Analysis “Risk Assessment, Economic Evaluation, and Decisions” workshop, September 26-27 2019

Evolving Science and Practice of Risk Assessment Katherine von Stackelberg, Harvard T.H Chan School of Public Health and NEK Associates LTD (kvon@nekassociates.com) and Pamela R.D. Williams, Colorado School of Public Health and E Risk Sciences LLP (pwilliams@erisksciences.com) The evaluation and management of public health risks relies largely on a risk assessment process defined by the regulatory context in which these risks are being assessed to answer a specific question. Although the science and practice of human health and ecological risk assessment has changed dramatically over the last 30 years and continues to evolve, the regulatory frameworks guiding the conduct of these assessments have not kept pace with such advances. For example, early applications of risk assessment were used to assess exposures and risks to specific chemicals at contaminated sites, leading regulatory agencies to back-calculate target remedial goals and clean-up levels on a chemical-by-chemical basis to answer the question “how clean is clean?” EPA’s Superfund program and development of water quality criteria reflect this regulatory scope. However, advances in science in technology have led to the development of new methods and tools that are capable of answering far more complex questions about the risks posed by chemicals and other stressors to human and ecological health. For example, toxicity assessments have transitioned from multi-year in vivo studies in rodents to high throughput in vitro bioassays in human cell cultures. But, little guidance is available on which analytical methods can be used to integrate concentration-response functions based on these data with exposure estimates to quantify health risks in ways useful to inform regulatory decision-making. Risk analysts now have the tools to combine aggregate exposure pathways with adverse outcome pathways to better integrate multiple chemical and non-chemical stressors to evaluate cumulative health risks, but how does that process inform site cleanup or remediation decisions? Scientific advances across the spectrum of risk-related disciplines (e.g., engineering, exposure science, toxicology, ecology) therefore provide better information to support risk-based decision-making, but at the same time, it is increasingly difficult to derive defensible health benchmarks or clean-up goals based on this new information. In this paper, we highlight several novel risk assessment approaches that have gained increasing momentum in the field of risk analysis and discuss scientific versus regulatory tensions in the application of these approaches for risk assessments of the future. 1.0 The Traditional Risk Assessment Context The earliest applications of risk assessment emerged from the discovery that exposure to environmental contaminants was likely associated with an increased probability of adverse health outcomes, and it wasn’t technically possible to entirely remove those exposures and resulting risks. Recognizing there would always be some level of residual risk led to the question: what is the appropriate concentration in particular environmental media that would

2

ensure a de minimis probability of adverse health outcomes associated with low-level, long-term exposure to specific contaminants? The first seminal publication that defined the field, the “Red Book” (NRC 1983), outlined a process (Figure 1) for risk assessment including hazard identification, dose-response assessment, exposure assessment, and risk characterization. This set of discrete steps was further codified in 1986 through the Superfund Public Health Evaluation Manual (EPA, 1986), a manual for the retrospective assessment of hazardous wa ste sites to inform remediation decisions. This “bottom-up” approach relied, and continues to rely, on a definition of exposure to environmental contaminants in terms of legal boundaries, e.g., property boundaries, and the potential for migration beyond the fenceline and subsequent exposures in local communities. Of course, environmental contaminants are not actually constrained by property boundaries, and this artificial designation has been a significant contributor to the current situation in applied risk assessment. One of the earliest risk assessment applications was at the Love Canal site, which became a high-profile poster child for environmental contamination. Although community groups were active in calling for response actions, the analytical process itself focused on hypothetical individual exposures rather than actual population exposures, which would involve epidemiologic studies that generally would be unlikely to have the power to detect measurable impacts. Consequently, the tension between predictive modeling starting with environmental releases and subsequent exposures and observational studies of exposed communities was established, leading to a clear delineation between these disciplines. Yet, it is important to recognize that a key aspect of the risk assessment process is to provide a quantitative linkage between measured (or modeled) contaminant concentrations and potential health outcomes.

Figure 1: The Risk Assessment Process from the "Red Book" (NRC 1983)

3

The risk assessment process was also quickly adopted to provide prospective analyses regarding proposed industrial facilities to evaluate the potential health impacts of expected emissions. Starting with stack tests, these analyses relied on a set of models to predict atmospheric transport, wet and dry deposition, uptake into plants, plant consumption by livestock, deposition onto water and uptake into aquatic food webs, and subsequent human exposure

across environmental media as shown in Figure 2. In both the retrospective (“how clean is clean”) and prospective (“how much is too much”) cases, risk assessment follows a chemical-by-chemical approach that was readily incorporated into regulatory decision-making with a variety of guidance documents at both the federal and state levels. A key assumption of regulatory applications is the idea of health conservatism, that is, analyses should incorporate a series of upper-bound assumptions across the suite of models used to predict potential outcomes in order to protect the most susceptible hypothetical individual (e.g., reasonable maximum exposure). In this chemical-by-chemical approach for a hypothetical individual, the underlying algebra to characterize risks is very simple as shown in Figure 3.

The ADD (average daily dose or exposed dose) is calculated using a variety of exposure models and generally takes the form shown in Figure 4. Measured data, exposure models, or a combination are used to predict the expected concentration in the environmental media and then combined with a series of exposure factors (i.e., intake rate, exposure duration, averaging time, body weight) to estimate the potential dose. Supporting

models for estimating environmental concentrations typically include fate and transport (e.g., movement of contaminants in ground water or air; contaminant uptake into fish tissue, etc.)

Figure 2: Example of a Conceptual Model of Exposure to Environmental Contaminants

Figure 3: Equations for Calculating the Incremental Lifetime Cancer Risk and Hazard Quotient

Figure 4: Equations for Estimating Exposure Doses

4

together with site-specific data to estimate environmental concentrations used as the starting point for calculating potential dose. In this simplified, individual chemical approach, it is straight-forward to rearrange the equations to solve for a target cancer risk or hazard quotient and obtain the associated concentration in the environmental media (e.g., “Concentration” in Figure 4) interpreted as a “safe level” either prospectively or retrospectively. From a regulatory perspective, this is an easily understandable calculation resulting in an environmental concentration deemed “acceptable” with respect to human health effects. Over time, our understanding of how individuals are exposed to contaminants, as well as the kinds of data available with which to evaluate how contaminants exert their effects has grown exponentially, and the emergence of “-omics” technologies across the spectrum (Figure 5) of exposure to dose-response has led to unprecedented increases in data and associated bioinformatics methods with which to support a true understanding of potential risks. Indeed, the US EPA in 2014 stated that “such large-scale knowledge creation was unimaginable 15 years ago” (US EPA 2014). This proliferation of data and methods has led to a disconnect with respect to the original mandate of risk assessment to provide a framework for establishing “safe” levels of environmental contaminants both prospectively and retrospectively and how these data can best support that goal. In the following sections, we explore the questions risk assessment was designed to answer from 1983 to the present day and how the evolution of the risk assessment process relates to regulatory mandates that appear insufficient for addressing future challenges. 2.0 Hazard Identification / Dose Response Assessment The dose-response assessment is a key part of the risk assessment process that up until recently relied on standardized multiyear in vivo studies, typically in rodents, to develop toxicity reference values (referred to as health reference values in NAS 2017) to quantify the relationship between exposures and effects. These studies generate toxicity reference values for non-cancer (e.g., Reference Dose for ingestion or RfD and Reference Concentration for inhalation or RfC) and carcinogenic (Cancer Slope Factor or CSF) outcomes that are then published in the US EPA’s Integrated Risk Information System (IRIS; www.epa.gov/iris) and used as regulatory thresholds for comparisons to predicted exposure doses (Figure 3). IRIS defines these reference values as estimates of exposures for a given duration to the human population (including susceptible subgroups) that are likely to be without an appreciable risk of adverse health effects over a lifetime.

Figure 5: Glossary of -Omics (Liu et al. 2019)

5

Traditional toxicity testing relies on exposing laboratory animals to a range of doses of a single environmental contaminant, and have historically not provided much information on mode-of-action or informed an understanding of the mechanisms by which contaminants exert their adverse effects. Consequently, extrapolation of toxicity values for use in human health risk assessment has traditionally incorporated the application of a series of uncertainty factors to account for this lack of knowledge. The primary sources of uncertainty include species-to-species extrapolation and differences in sensitivity to response with the goal of protecting potentially susceptible or highly sensitive individuals. As shown in Figures 3 and 4, toxicity or health reference values derived in this way are easily incorporated into rearranged risk equations to solve for environmental concentrations with a clear regulatory interpretation. However, traditional in vivo toxicity testing is time-consuming, resource-intensive, doesn’t typically provide insight into mechanisms or mode-of-action, and cannot possibly address the tens of thousands of chemicals in active commerce. In the context of a population of hypothetical individuals, environmental exposures become increasingly complicated to model given the range of co-exposures, co-morbidities, and potential susceptibilities that exist across the population. In the additive model, exposures to individual chemicals are assumed as independent, with neither synergistic (e.g., greater than additive) or antagonistic (e.g., ameliorating effects) interactions. Evidence has emerged that under certain exposures this is a false assumption, with ramifications particularly in the case of synergistic outcomes (von Stackelberg et al. 2015). Gene-environment interactions also impact the probability of an effect occurring. 2.1 Systems Toxicology

Systems toxicology focuses on understanding biological pathways by which a series of biological events lead to one or more adverse outcomes in an organism (Aguayo-Orozco et al. 2019). The adverse outcome pathway (AOP) is a conceptual framework for this process as shown in Figure 6, starting with a molecular initiating event (MIE) followed by a series of key events (KEs) ultimately leading to one or more outcomes of regulatory interest. Contaminant exposures can engage at any point in this process; that is, there can be any number

of macromolecular biological interactions including transcript alterations, gene and protein expression, metabolic pathways, and epigenetics. Organism responses can also range from resilience and adaptation to an adverse state. While AOPs are typically described as linear, in fact exposure to multiple contaminants can lead to perturbations across multiple pathways and

Figure 6: Example of an Adverse Outcome Pathway

6

biological mediators, effectively a network of pathways interacting in different ways with different activation thresholds across the human population. Through an international effort led by the Organisation for Economic Co-operation and Development (OECD), an AOP wiki (www.aopwiki.org) has been established for the vetting and dissemination of AOPs, and which aims to develop guidance detailing internationally accepted approaches for describing and documenting AOPs for potential regulatory applications. As stated by OECD, AOPs are intended “to outline and capture existing knowledge concerning the biologically plausible and empirically supported foundations for predicting apical toxicity from mechanistic data” (OECD 2016). However, in general, as shown in Figure 6 for acetylcholinesterase inhibition, AOPs tend to be linear and provide qualitative descriptions of MIEs and KEs rather than quantitative relationships. A key advance in the risk assessment process is the transition from in vivo toxicity tests to a host of in vitro and in silico assays, including high throughput screening (HTS) and a variety of “-omics” (Figure 5) to characterize the relationship between concentration or dose and effect. The ToxCast program developed by the US EPA (https://www.epa.gov/chemical-research/toxcast-data-generation-toxcast-assays) uses automated chemical screening technologies, called high-throughput screening assays, to expose living cells or biological macromolecules such as proteins to contaminants. The cells or isolated proteins are then screened for changes in biological activity that may indicate potential health hazards in living systems. ToxCast provides contaminant-specific data based on over 700 high-throughput assays that cover a range of high-level cell responses and approximately 300 signaling pathways. Cell-based in vitro assays use intact cells to measure cellular changes in response to exposure with the test substances, while biochemical in vitro assays measure the activity of a biological macromolecule, either a purified protein or a nucleic acid. These data can be used to backcalculate in vivo concentrations associated with in vitro bioassay results (Sipes et al. 2017) as shown in Figure 7. These approaches utilize pharmacokinetic equations to estimate chemical steady-state concentrations (Css) in plasma and to dosimetrically adjust in vitro HTS potencies to estimate external dose equivalents (intake rates). An open-source R package, High-Throughput Toxicokinetic (HTTK), calculates equivalent external doses from HTS data using multiple IVIVE models and, conversely, calculates plasma concentrations given particular dosing scenarios. With the advent of bioinformatics and machine learning, these kinds of data allow for rapid screening of large numbers of contaminants (Shin et al. 2015; Rotroff et al. 2010). In principle,

Figure 7: Schematic for the HTTK R-Package (Sipes et al. 2017)

7

individual bioassay results could be mapped to MIEs and KEs across different AOPs (Wittwehr et al. 2017), but this is challenging in practice since these individual results cannot capture the complexity of living systems in terms of feedback loops, physiological responses, and interactions. In one of the first examples of a quantitative AOP, Zgheib et al. (2019) developed a Bayesian Network model to quantify a simplified oxidative stress induced chronic kidney disease AOP based on in vitro data obtained by exposing hepatic cells to potassium bromate. Transcriptomic and toxicogenomic data, such as differential gene expression profiling, pathway analysis, genome sequence analysis, proteomics, metabolomics, and related approaches enable the characerization of transcriptomic signatures for specific types of chemical-induced toxicities (Vinken et al. 2019; Liu et al. 2019). Similarly, the goal is to map these signatures to KEs and MIEs in AOPs. There are now numerous public databases about chemicals, genes, proteins, biological networks, phenotypes, diseases, and exposure science that can be integrated to construct pathways for systems toxicology applications and many of these are described in Davis et al. (2019). Further efforts are required to account for variables in test systems that can influence observed associations between molecular perturbations and disease outcomes (e.g., experimental design, metabolism, genomic variants, target cell type(s), cell and tissue interactions, species, and lifestage). Ongoing studies seek to cross-validate in vitro test results (Martin et al. 2011), but one cross-validation study indicated that the current ToxCast phase I assays and chemicals have limited applicability for predicting in vivo chemical hazards using standard statistical classification methods (Thomas et al. 2012). Although these methods offer the promise of a better mechanistic understanding of contaminant toxicity, the question is, how will these approaches be incorporated into and inform regulatory decision-making ? 3.0 Exposure Assessment A critical step in this process is evaluating potential exposures based on a conceptual model such as the one shown in Figure 2. In a focused decision-making context, for example, impacts associated with power plants, or, even more focused, mercury concentrations associated with electricity generation, very specific and reductionist conceptual models of exposure support a series of quantitative models to predict the ADD for a hypothetical individual. This hypothetical individual context allows for an abstraction of exposure such that it is unclear how model results relate to true exposures in the population. As our understanding of exposure continues to grow, a number of new concepts, methods, and tools have emerged to better capture and reflect this increased knowledge. 3.1 Biomonitoring Biomonitoring provides a direct measure of total exposure from all sources and exposure routes and pathways at a given time period. Biomonitoring data generally represent the concentration of a chemical or its metabolite as measured in biological matrices such as blood,

8

urine, hair, nails, fat, bone, expired air, or breast milk and provide a biologic indicator (or marker) of exposure – i.e., the underlying concept is that chemicals that have entered the human body leave “markers” reflecting this exposure (Sexton et al. 2004; NRC 2006; US EPA 2012). These markers of exposure generally represent absorbed or internal doses, but may also represent biologically effective doses or early biological effects (Sexton et al. 2004; NRC 2006; US EPA 2012). Although many resources and guidelines are available for the proper collection of biological samples (Cornelis 1995; 1996), fewer guides are available on when to collect and how to interpret biomonitoring data for use in exposure and risk assessments. Over the last 20 years, the Centers for Disease Control and Prevention (CDC) has published the results of ongoing surveys that characterize the distribution of exposures by age and sex groupings for different chemicals in biological matrices as part of the National Report on Human Exposures to Environmental Chemicals, but with little accompanying interpretation (www.cdc.gov/biomonitoring). Many publications have also reported on biomonitoring data collected as part of specific research studies designed to assess public exposures (Aylward 2018). NRC (2006) has recommended that a more coordinated public health-based strategy for biomarker development is needed and that exposure, epidemiology, and toxicology studies should be enhanced with biomonitoring wherever possible to in order to better interpret the risks posed by low-level exposures to environmental chemicals. In response to these recommendations, US EPA (2012) has outlined its research strategies for improving the use and interpretation of biomonitoring and biomarker data in human exposure and risk assessments as shown in Figure 8, which includes five research tiers depending on the research objectives and available data and models. Some attempt has also been made to develop “common criteria” to inform the use and evaluation of biomonitoring data used in exposure and human health risk assessments (Albertini et al. 2006; Arnold et al. 2013). Other research has focused on deriving biological exposure indices (BEIs) or biomonitoring equivalents (BEs) that can serve as relevant health benchmarks for interpreting biomonitoring data (ACGIH 2012; Hays et al. 2007; Hays and Aylward 2009).

9

Biomonitoring has many useful purposes and can serve as a powerful tool for assessing aggregate exposures and risks, particularly for some common chemicals. Biomonitoring data can be used to establish baseline exposure levels in a population, assess trends in exposure over time, validate environmental and exposure models, and evaluate the effectiveness of risk management programs. Two well-known examples are the use of biomonitoring data to (a) illustrate how blood lead levels declined as gasoline lead levels decreased over time, and (b) demonstrate reductions in serum cotinine levels among non-smokers due to smoking cessation efforts (Sexton et al. 2004). However, collecting biological samples and determining which biomarkers

to analyze for can be a challenging and labor- and resource-intensive process. Biomonitoring data also only provide a measure of exposure, without any indication of whether such exposures pose a public health risk. Additionally, because biomonitoring data represent a measure of internal body burden, the original sources or routes and pathways of exposure and exposure conditions (e.g., concentration, frequency and duration) may be variable or unknown, rendering it difficult to design effective exposure reduction or mitigation policies. Linkages between biomarkers of exposure and downstream health effects are also lacking. Given these and other constraints, biomonitoring data have thus far had limited use in regulatory risk assessments. In particular, EPA generally only uses biomonitoring data in chemical risk assessments when there is sufficient research showing the health implications of these data. For example, when evaluating the public health risks from methylmercury, US EPA (2001) relied on certain biomonitoring data (i.e., mercury in maternal hair collected at delivery and mercury in umbilical cord blood) because existing studies had already drawn a link between the level of this toxin in human blood and adverse neurological effects in children. Going forward, how can this type of information be better utilized by the US EPA or other public health and regulatory agencies to assess and manage public exposures and risks? 3.2 Exposome The exposome is a measure of the totality of exposure (both internal and external) over a lifetime, with an emphasis on understanding how such exposures ultimately impact public health. The concept of the exposome originally stemmed from the desire to provide more complete exposure assessments in epidemiology studies and to achieve a comprehensive description of lifelong exposure history as a complement to the genome (Wild 2005, 2011). The underlying premise of the exposome is that an individual’s exposure begins before birth and includes insults from environmental and occupational sources, with possible gene-environment

Figure 8: The Source-to-Outcome Continuum to Support Human Health Research (US EPA 2012)

10

interactions occurring throughout one’s lifetime (Wild 2005; Rappaport and Smith 2010; Buck Louis and Sundaram 2012). Although initial discussions about the exposome primarily targeted the field of exposure science and focused on human health impacts, definitions and intended uses of the exposome have varied and evolved over time. NRC (2012) proposed to expand this concept to the “eco-exposome,” which entails the “extension of exposure science from the point of contact between stressor and receptor inward into the organism and outward to the general environment, including the ecosphere” (p. 32). Miller and Jones (2013) proposed the following broader definition that implicitly considers both exposures and biological responses: “a cumulative measure of environmental influences and associated biological responses throughout the lifespan, including exposures from the environment, diet, behavior, and endogenous processes” (p. 2). Current and ongoing studies and research initiatives related to the exposome have included a strong clinical component, including the collection of omics data, to better assess potential linkages between the exposome and adverse health outcomes (Vrijheid et al. 2014; Maitre et al. 2018).

Resources and guidance documents pertaining to the exposome and how to measure it are somewhat sparse, although a variety of studies and research initiatives are underway in the United States and Europe. From an exposure standpoint, proposed strategies for characterizing an individual’s exposome generally include a “top down” versus “bottom up” approach – the former consists of first collecting biomonitoring data and then identifying all important

exposures, whereas the latter consists of first assessing chemical concentrations in the environment (e.g., air, water, food) and then identifying important exogenous exposures (Rappaport 2011). A variety of measurement tools are currently being investigated, developed, or refined to quantify the exposome as shown in Figure 9, including conventional survey instruments, personal sensors, geographic information systems, biomarker measurements, and -omic technologies (Figure 5) (Nieuwenhuijsen et al. 2014; DeBord et al. 2016). Since 2013, the National Institute of Environmental Health Sciences (NIEHS) has also funded an exposome research center at Emory University called HERCULES, which has already published research findings on a range of topics including exposure assessment, systems biology, epigenetics, and computational systems analysis (https://tools.niehs.nih.gov/portfolio/index.cfm/portfolio/ grantDetail/ grant_number/P30ES019776). Additionally, a collaborative research project in Europe called HELIX involves six longitudinal birth cohort studies, and preliminary baseline data

Figure 9: Conceptual Framework of Selected Scientific and Technological Advances as Described in NRC 2012

11

have been obtained in an attempt to characterize early-life exposure to multiple environmental factors and associate these with omics biomarkers and child health outcomes (Vrijheid et al. 2014). The exposome holds much promise for eventually being able to characterize complex mixtures of exposure over time, particularly during vulnerable life-stages, and establishing linkages between exposures and health outcomes. This approach may also spur new technical and scientific developments and yield a better understanding of important gene-environment interactions. However, mapping an entire exposome is challenging and will require substantial data collection efforts and sophisticated data analyses over a longer time horizon. Appropriate measurement techniques will also need to be vetted and validated. Of even greater importance, it may not be possible to actually regulate or manage identified exposures or gene-environment interactions on a population level, due to the inherent variability among individuals’ exposomes, genetics, and other personal risk factors. Assuming individual exposomes can be accurately measured and linked to individual health outcomes, how will this information be used by public health or regulatory agencies to reduce or manage public exposures and risks, or alternatively, will this information only be useful for personalized medicine? For example, consider the following scenario: it is determined that individual #1 is exposed to chemical A in air, which in combination with that individual’s genetic makeup and other personal risk factors, leads to disease X, whereas individual #2 is exposed to chemical B in drinking water, which in combination with that individual’s genetic makeup and other personal risk factors, leads to disease Y. Considering an entire population with varying individual exposure profiles, diverse genetics and other personal risk factors, and different disease patterns, what can or should public health and regulatory agencies do to manage environmental contaminants? 3.3 Aggregate Exposure Pathway The concept of the aggregate exposure pathway (AEP) is similar to the Exposome and has been proposed as a framework for organizing and integrating diverse exposure information that exists across numerous repositories and among multiple scientific fields (Tan et al. 2018a; 2018b; Esche et al. 2017). Teeguarden et al. (2016) proposed the concept of the AEP as the natural and complementary companion in the exposure sciences to the AOP concept in the toxicological sciences. An AEP is the assemblage of existing knowledge concerning biologically, chemically and physically plausible, empirically supported links between environmental concentrations of contaminants and concentrations at a site of action, i.e. target site exposure, which could include protein, cell, organelle, tissue, or organ. Similar to AOPs, the concept of the AEP proposes a linear pathway from source (e.g., industrial release, waste water, etc.) to exposure medium, to external followed by internal exposure, and finally target site exposure. The target site exposure then links directly to a MIE or KE in an AOP. The emerging discipline of computational exposure science identifies new data sources and the application of innovative modeling techniques for understanding and quantifying human exposures to chemicals (Egeghy et al. 2016; Cohen Hubal et al. 2010).

12

4.0 Risk Characterization

Risk characterization is the process of combining an exposure estimate with a dose-response function to calculate the probability of a carcinogenic outcome, or exceedance of a health-based threshold (hazard quotient) as shown in Figure 4. Probabilistic methods can be used to predict distributions of exposure estimates, but use of probabilistic approaches is discouraged for toxicity assessments. The hazard quotient (e.g., comparison of a predicted exposure dose or air concentration to a RfD or RfC, respectively) is not a particularly informative metric as an exceedance (e.g., hazard quotient greater than one) offers no probabilistic interpretation. 4.1 Cumulative Risk Assessment Cumulative risk assessment (CRA) involves evaluating the combined risk from co-exposure to multiple chemical and non-chemical stressors that have the same health effect. A primary impetus for this approach was the recognition the vulnerable groups, such as infants and children, may be co-exposed (via various sources and exposure pathways) to multiple stressors that share a common toxic effect (NRC 1993). Possible initiating factors for conducting a CRA in any population as shown in Figure 10 include (a) the identification of multiple sources or exposures or (b) an observed increase in symptoms or illnesses. CRAs represent a shift from traditional chemical-based assessments, which typically start with the source or release of a chemical and evaluate how the chemical moves through the environment and contacts target receptors or populations, to a more population-based assessment, whereby the receptors or populations of interest are the starting point and a determination is made about which chemicals, non-chemical stressors, or other risk factors affect these groups (EPA 2003, 2007). A number of resources and guidance documents are available that describe the CRA framework and steps for analysis. For example, the EPA has authored several publications related to assessing aggregate and cumulative risks, particularly for pesticides (EPA 2001b, 2002, 2003, 2007). The EPA has relied on this guidance to conduct quantitative CRAs for the following five different classes of pesticides: organophosphates, carbamates, triazines, chloroacetanilides, and pyrethrins/pyrethroids (https://www.epa.gov/pesticide-science-and-assessing-pesticide-risks/cumulative-assessment-risk-pesticides). Additionally, many professionals and practitioners have published overviews of the CRA approach and existing tools, with an emphasis on how this methodology is advantageous

Figure 10: Example Initiating Factors and Data Elements for Cumulative Risk Analyses (US EPA 2007)

13

for assessing health risks in environmental, community, or occupational contexts (Callahan and Sexton 2007; Menzie et al. 2007; Barzyk et al. 2010; Sexton and Linden 2011; Williams et al. 2012; Macdonell et al. 2013; Gallagher et al. 2015; Lenz et al. 2015; Fox et al. 2017; Moretto et al. 2017). Some research has also focused on specific scientific and technical issues associated with conducting CRAs, such as integrating chemical and non-chemical stressors and accounting for toxicological interactions among stressors (Lewis et al. 2011; Sexton et al. 2012; Rider et al. 2014). NRC (2009) has recommended the continued development of CRA methods and tools that can be used to support health risk assessments in the future (NRC 2009). The CRA framework provides a useful approach for more holistically evaluating combined exposures and risks, which may result in an additive or greater-than-additive (e.g., synergistic) effect. Personal risk factors that can modify the risk, such age, sex, behavioral/lifestyle patterns or activities, or comorbidities, can also be assessed under this approach (Schulte et al. 2012). However, despite CRA’s potential promise for answering more complex questions about combined risks, there are many data gaps and methodological challenges that remain. For example, there is currently no consensus and limited practical guidance on how to how to identify, group, and quantify co-exposures to different stressors across different domains; how to combine different types of stressor exposures using the same currency or unit of measurement; how to characterize or account for possible toxicological interactions; or how to derive dose-response or toxicity criteria values for different co-exposure combinations (Maier et al. 2017). The lack of health benchmarks and constrained regulatory purview also limit the ability to consider and implement effective risk management options. That is, even if one could accurately identify and quantify relevant combined risk scenarios, how will this information be used by EPA or other public health and regulatory agencies to reduce or manage public exposures and risks? For example, consider the following scenario: it is determined that a population is simultaneously exposed to chemical A in air, chemical B in drinking water, non-chemical stressor C, and personal risk factor D, and all of these exposures contribute to the same health effect. What can or should public health and regulatory agencies do in this situation? 5.0 Ecological Risk and Ecosystem Services It was not until the early 1990’s (US EPA, 1994) that the concept of ecological risk assessment (ERA) emerged as a way to evaluate potential outcomes in wildlife associated with exposure to environmental contaminants. Early ERAs mirrored the methodology followed by under non-cancer human health risk assessment, namely comparing predicted exposure doses in mg contaminant per kg of body weight per day to threshold values derived from either field or laboratory studies. Exposure data compiled in the Wildlife Exposure Factors Handbook (US EPA, 1993) followed the same approach as the human health-based Exposure Factors Handbook (US EPA, ). However, unlike for human health for which contaminant-specific toxicity reference values are vetted through IRIS, there is significant leeway in determining the most appropriate threshold value for comparison to predicted doses. Also unlike human health risk assessment, there is no regulatory barrier to using probabilistic approaches for both exposure and dose-response, yet in practice, true probabilistic assessments are few and far between.

14

A key difference between human health and ecological risk assessment is the distinction between measurement endpoints and assessment endpoints since in ecological risk assessment, the focus is not on hypothetical individuals but rather populations of animals. Assessment endpoints are defined as the ecological endpoints of regulatory interest, for example, sustainability of wildlife populations such as mink or deer. Since it is difficult to observe impacts at a population level, particularly in the prospective risk assessment context, measurement endpoints tend to focus on individuals. An important aspect of ERA, though, is that it provides a fundamental framework for evaluating the impact of stressors, including contaminants, on the natural environment. In the original formulation of ERA, animal populations are considered as analogues to the hypothetical individual human that is the focus of conventional HHRA. Rather than focus on particular animal populations in a reductionist way, the concept of ecosystem services has emerged as a way to frame the benefits of the natural environment on human well-being in an effort to justify management activities that shape decisions on land use, resource extraction, manufacturing, and trade so that the widespread declines in the ecosystem-rooted life-support systems can be arrested or reversed. (NRC 2012b; Munns et al. 2016a). “Sustainability” has become a central force and focus in decision-making, even as environmental degradation as a consequence of human activities continues at unprecedented rates worldwide. Ecosystem services provide the framework and language for documenting the direct and indirect links between human well-being and the many benefits provided by the natural systems humans occupy (NRC 2012b), and represent a logical bridge in coupled human-natural systems thinking and analyses. ERA is a logical framework for incorporating the concept of ecosystem services as an explicit measure of ecosystem health, and more recently, this concept has gained traction with the US EPA publication of generic ecological assessment endpoints framed in terms of ecosystem services (US EPA 2016; Munns et al. 2016a; 2016b). Under this approach, specific ecological receptors are less important than the overall ecosystem goals, with an explicit recognition of the importance of the natural environment to human well-being. Ecosystem services provide a direct linkage between HHRA and ERA, particularly through the use of decision analytic approaches (von Stackelberg 2013). Despite numerous international efforts to quantify and incorporate ecosystem services into decision making, little has changed. Human health is inextricably linked to ecosystem health, but how will regulatory programs effectively use systems thinking to support decision making? 6.0 The Path Forward Figure 11 is taken from US EPA’s proposed Next Generation Risk Assessment (2014) and describes a much more nuanced, decision-focused approach for conducting risk assessments based on an integration of newer data and methods. The original process shown in Figure 1 now begins with a much more detailed consideration of objectives under problem formulation

15

and scoping that takes into consideration the risk context, decision-making options, and potential for value of information. This initial phase is followed by the actual risk assessment using a population health approach to identify multiple health determinants and how they interact with the risk factor(s) of interest. Under this model, hazard identification, dose–response assessment, and exposure assessment (and risk characterization?) rely on emerging scientific tools and technologies, including high throughput screening assays and computational methods for systems toxicology. As discussed previously, dose- or concentration-response assessment relies on in vitro to in vivo extrapolation methods for calibration of in vitro and human dosimetry and a host of molecular and genetic epidemiologic approaches to identify toxicity pathway perturbations in population-based studies to develop formal AOPs. In terms of exposure assessment, AEPs combined with biomonitoring data are used to characterize human exposure at the population level. Finally, the risk

management phase integrates these results into a risk-based decision-making approach that incorporates economic analyses, sociopolitical considerations and community risk perception. This is not a framework that easily lends itself to answering simple questions on an individual chemical basis, such as “how clean is clean” or “how much is too much.” The regulatory need for threshold values for individual contaminants protective of public health is easily met using risk assessment methods based on simple algebraic functions. However, the true level of protection is unknown given the complexity of combined exposures, susceptibilities, and genetic composition. Bioinformatics and systems toxicology provide ever more information and data with which to develop dose-response or concentration-response functions for macromolecular interactions and cellular to organ to whole body events along AOPs. Epidemiologic methods to explore relationships between exposure metrics and biomarkers of effect also provide perspective on the potential for adverse effects, but these data are typically not used in regulatory risk assessments as currently practiced. Similarly, on the exposure side, biomonitoring data provide evidence of combined exposures from all sources and absorbed concentrations of contaminants. Combining those approaches and underlying analyses as shown in Figure 11 answers a different set of questions, but ones likely more relevant to societal well-being. With the number of contaminants in the tens of thousands (for example, there are over 4,500 known per- and polyfluorinated compounds alone, and biomonitoring data show that virtually every human and animal have measurable amounts of these contaminants), it is unlikely the

Figure 11: Next Generation Risk Assessment Framework (US EPA 2014)

16

single chemical individual threshold model will be protective, and clearly it will be impossible to derive toxicity reference values for every contaminant. Even if such levels could be derived, the combined exposure across so many contaminants in the context of other stressors (e.g., socioeconomics, exposure to the natural environment, climate change, etc.) leads to a “death by a thousand cuts” situation. The original regulatory mandates, “how clean is clean” and “how much is too much” seem far less relevant and poorly supported by the kind of information currently being generated. The original simplified (largely deterministic) risk assessment process is fundamentally a reductionist approach that quantifies chemical-by-chemical exposures for a hypothetical individual. The evolution of risk assessment to better understand how the complexity of external exposures to not just contaminants but other stressors, internal genomic makeup, and disease etiology combine to influence well-being at the population level rather than the hypothetical individual level provides a much more comprehensive framework for evaluating contaminant exposures. For example, Hines et al. (2017) present an example case study combining an AEP network (e.g., combined exposures) with an AOP network (e.g., mechanistic dose-response assessment) to demonstrate an integrated HHRA/ERA to predict cumulative risks across the population (Figure 12). Looking at Figure 12, it is difficult to relate the outcomes of this process in the context of remediation decisions or permitting requirements. However, it is likely a useful framework for identifying – out of tens of thousands of contaminants – a structured and data-informed prioritization of which contaminants to focus on from a regulatory perspective. The emphasis now is on developing reliable molecular indicators or biomarkers of exposure and effects across a wide variety of contaminants to better leverage the biomonitoring and clinical data generated at the population level, and combining those data with the bioinformatics and -omics data from in vitro and in silico bioassays. These large datasets are being used to construct and understand key pathways in the network of interactions among genes, cells, macromolecular tissues, and organs to move from chemical-by-chemical dose response functions to true predictive toxicology. Another component to this well-being perspective is characterizing population variability and how genetic composition, preexisting backgrounds of disease and exposure, including epigenetics, and adaptive or compensatory processes combine to influence population risks.

Figure 12: Integrated AEP-AOP Network (Hines et al. 2017)

17

But there is potentially a bigger challenge. A 2012 report from the NRC (NRC 2012c) presents a holistic and inclusive framework for enhanced science for environmental protection at a very fundamental level. As shown in Figure 13, the panel proposed an iterative process that begins with comprehensive problem formulation, similar to Figure 12, to identify solution-focused policy and decision goals. Data and knowledge are then acquired and synthesized to generate insight around key outcomes using an array of systems tools and synthesis approaches with the goal of developing policies and decisions

that best improve public health and the environment. In this holistic view, risk assessment becomes one of the tools in a set of fit-for-purpose approaches to inform decision-making from a systems perspective (NRC 2009). It seems clear that risk assessment as a process and framework is moving away from “safe levels” and toward a better understanding of how aggregated and lifetime environmental exposures impact humans in the context of genetic predisposition and makeup. Contaminants in the environment are fundamentally an environmental problem and represent but one of the many human-induced factors contributing to widespread environmental degradation, which in turn influences human well-being. The recognition that humans operate within the biosphere and are an inextricable component of a coupled human-natural system sets up a broader decision context. Regulatory questions focused on individual contaminant concentrations fail to capture this larger perspective, and fail to take advantage of our improved understanding of the relationship between the environment writ large and human well-being. To enable efficient and cost-effective public policy in the future, we will need to reconcile the tensions between increasing scientific knowledge and regulatory actions. Can improvement to the risk assessment process be the answer? 7.0 References Aguayo-Orozco A, Taboureau O, Brunak S. 2019. The use of systems biology in chemical risk assessment. Current Opinion in Toxicology. 2019 Mar 12. Albertini R, Bird M, Doerrer N, et al. 2006. The use of biomonitoring data in exposure and human health risk assessments. Environmental Health Perspectives. 114:1755–1762.

Figure 13: Schematic for Addressing Future Environmental Challenges (NRC 2012c)

18

ACGIH. 2012. Documentation of the Threshold Limit Values and Biological Exposure Indices. American Conference of Governmental Industrial Hygienist. Cincinnati, OH. Arnold SM, Angerer J, Boogaard PJ, Hughes MF, O’Lone RB, Robison SH, Schnatter AR. 2013. The use of biomonitoring data in exposure and human health risk assessment: benzene case study. Critical Reviews in Toxicology. 43:119–153. Aylward LL. 2018. Integration of biomonitoring data into risk assessment. Current Opinion in Toxicology. 9:14-20. Barzyk TM, Conlon KC, Chahine T, Hammond DM, Zartarian VG, Schultz BD. 2010. Tools available to communities for conducting cumulative exposure and risk assessments. Journal of Exposure Science and Environmental Epidemiology. 20: 371–384. Buck Louis GMB, Sundaram R. 2012. Exposome: time for transformative research. Statistical Medicine. 28:31(22). Callahan MA, Sexton K. 2007. If cumulative risk assessment is the answer, what is the question? Environmental Health Perspectives. 115:799-806. Cohen Hubal EA, Richard A, Aylward L, Edwards S, Gallagher J, Goldsmith MR, Isukapalli S, Tornero-Velez R, Weber E, Kavlock R. 2010. Advancing exposure characterization for chemical evaluation and risk assessment. Journal of Toxicology and Environmental Health, Part B. 13(2-4):299-313. Cornelis R, Heinzow B, Herber RF, Christensen JM, Poulsen OM, Sabbioni E, Templeton DM, Thomassen Y, Vahter M, Vesterberg O. 1995. Sample collection guidelines for trace elements in blood and urine (technical report). Pure and Applied Chemistry. 67(8-9):1575-608. Cornelis R, Heinzow B, Herber RF, Christensen JM, Poulsen OM, Sabbioni E, Templeton DM, Thomassen Y, Vahter M, Vesterberg O. 1996. Sample collection guidelines for trace elements in blood and urine. Journal of Trace Elements in Medicine and Biology. 10(2):103-27. Davis AP, Wiegers J, Wiegers TC, Mattingly CJ. 2019. Public data sources to support systems toxicology applications. Current Opinion in Toxicology. 2019 Mar 11. DeBord DG, Carreón T, Lentz TJ, Middendorf PJ, Hoover MD, and Schulte PA. 2016. Use of the “Exposome” in the practice of epidemiology: a primer on –omic technologies. American Journal of Epidemiology. 184:302–314. Egeghy PP, Sheldon LS, Isaacs KK, Özkaynak H, Goldsmith MR, Wambaugh JF, Judson RS, Buckley TJ. 2015. Computational exposure science: an emerging discipline to support 21st-century risk assessment. Environmental Health Perspectives. 124(6):697-702.

19

Escher BI, Hackermüller J, Polte T, Scholz S, Aigner A, Altenburger R, Böhme A, Bopp SK, Brack W, Busch W, Chadeau-Hyam M, Covaci A, Eisentrӓger A, Galligan JJ, Garcia-Reyero N, Hartung T, Michaela Hein M, Herberth G, Jahnke A, Kleinjans J, Klüver N, Krauss M, Lamoree M, Lehmann I, Luckenbach T, Miller FW, Müller A, Phillips DH, Reemtsma T, Rolle-Kampczyk U, Schüürmann G, Schwikowski B, Tan Y-M, Trump S, Walter-Rohde S, Wambaugh JF. 2017. From the exposome to mechanistic understanding of chemical-induced adverse effects. Environment International. 99:97-106. Fox MA, Brewer LE, Martin L. 2017. An overview of literature topics related to current concepts, methods, tools, and applications for cumulative risk assessment (2007-2016). International Journal of Environmental Research and Public Health. Apr 7;14. Gallagher SS, Rice GE, Scarano LJ, Teuschler LK, Bollweg G, Martin L. 2015. Cumulative risk assessment lessons learned: a review of case studies and issue papers. Chemosphere. 120:697-705. Hays SM, Becker RA, Leungc HW, Aylward LL, Pyatt DW. 2007. Biomonitoring equivalents: A screening approach for interpreting biomonitoring results from a public health risk perspective. Regulatory Toxicology and Pharmacology. 47:96–109. Hays SM, Aylward LM. 2009. Using biomonitoring equivalents to interpret human biomonitoring data in a public health risk context. Journal of Applied Toxicology. 29:275–288. Hines DE, Edwards SW, Conolly RB, Jarabek AM. 2017. A case study application of the aggregate exposure pathway (AEP) and adverse outcome pathway (AOP) frameworks to facilitate the integration of human health and ecological end points for cumulative risk assessment (CRA). Environmental Science & Technology, 52(2):839-849. Lentz TJ, Dotson GS, Williams PR, Maier A, Gadagbui B, Pandalai SP, et al. 2015. Aggregate Exposure and Cumulative Risk Assessment--Integrating Occupational and Non-occupational Risk Factors. Journal of Occupational and Environmental Hygiene. 12 Suppl 1:S112-26. Lewis AS, Sax SN, Wason SC, Campleman SL. 2011. Non-chemical stressors and cumulative risk assessment: an overview of current initiatives and potential air pollutant interactions. International Journal of Environmental Research and Public Health. 8:2020-73. Liu Z, Huang R, Roberts R, Tong W. 2018. Toxicogenomics: a 2020 vision. Trends in Pharmacological Sciences. 2018 Dec 26. Macdonell MM, Haroun LA, Teuschler LK, Rice GE, Hertzberg RC, Butler JP, et al. 2013. Cumulative risk assessment toolbox: methods and approaches for the practitioner. Journal of Toxicology. http://dx.doi.org/10.1155/2013/310904.

20

Maier A, Williams PR, Dotson GS. 2017. Risk in Combination- Addressing Cumulative Risk in the Occupational Environment. The Synergist. Maitre L, de Bont J, Casas M, Robinson O, Aasvang GM, et al. 2018. Human Early Life Exposome (HELIX) study: a European population-based exposome cohort. BMJ Open. 8:e021311. Martin MT, Knudsen TB, Reif DM, Houck KA, Judson RS, Kavlock RJ, Dix DJ. 2011. Predictive model of rat reproductive toxicity from ToxCast high throughput screening. Biology of Reproduction. 85(2):327-39. Menzie CA, MacDonell MM, Mumtaz M. 2007. A phased approach for assessing combined effects from multiple stressors. Environmental Health Perspectives. 115(5):807-816. Miller GW, Jones DP. 2013. The nature of nurture: refining the definition of the exposome. Toxicological Sciences. 137: 1–2. Moretto A, Bachman A, Boobis A, Solomon KR, Pastoor TP, Wilks MF, et al. 2017. A framework for cumulative risk assessment in the 21st century. Critical Reviews in Toxicology. 47:85-97. Munns Jr WR, Rea AW, Suter GW, Martin L, Blake-Hedges L, Crk T, Davis C, Ferreira G, Jordan S, Mahoney M, Barron MG. 2016a. Ecosystem services as assessment endpoints for ecological risk assessment. Integrated Environmental Assessment and Management. 13(3):522-8. Munns WR, Poulsen V, Gala WR, Marshall SJ, Rea AW, Sorensen MT and K von Stackelberg. 2016b. Ecosystem services in risk assessment and management. Integrated Environmental Assessment and Management. 13(1):62-73. DOI: 10.1002/ieam.1835. NAS, National Academies of Sciences, Engineering, and Medicine. 2017. Using 21st Century Science to Improve Risk-Related Evaluations. Washington, DC: The National Academies Press. doi: 10.17226/24635. NRC, National Research Council. 1983. Risk assessment in the federal government: managing the process. Washington, DC: National Academies Press. NRC, National Research Council. 1993. Pesticides in the Diets of Infants & Children. National Research Council, National Academy of Sciences. Washington, DC: National Academies Press. NRC, National Research Council. 2006. Human Biomonitoring for Environmental Chemicals. National Academy of Sciences. Washington, DC: National Academies Press. NRC, National Research Council. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: National Academies Press.

21

NRC, National Research Council. 2012a. Exposure Science in the 21st Century: A Vision and a Strategy. Committee on Human and Environmental. Washington, DC: The National Academy Press. NRC, National Research Council. 2012b. Ecosystem Services: Charting a Path to Sustainability: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. https://doi.org/10.17226/13331. NRC, National Research Council. 2012c. Science for Environmental Protection: The Road Ahead. Washington, DC: The National Academies Press. https://doi.org/10.17226/13510. NRC, National Research Council. National Academies of Sciences, Engineering, and Medicine. 2017. Using 21st Century Science to Improve Risk-Related Evaluations. Washington, DC: The National Academies Press. https://doi.org/10.17226/24635. Nieuwenhuijsen MJ, Donaire-Gonzalez D, Foraster M, Martinez D, Cisneros A. 2014. Using personal sensors to assess the exposome and acute health effects. International Journal of Environmental Research and Public Health. 11:7805-7819 OECD, Organisation for Economic Co-operation and Development (OECD). 2016. Guidance Document for the Use of Adverse Outcome Pathways in Developing Integrated Approaches to Testing and Assessment (IATA). Series on Testing and Assessment, No. 260, ENV/JM/MONO (2016) 67, OECD Environment, Health and Safety Publications. Rappaport SM, Smith MT. 2010. Environment and disease risks. Science. 330:460–461. Rappaport SM. 2011. Implications of the exposome for exposure science. Journal of Exposure Science and Environmental Epidemiology. 21:5–9. Rider CV, Boekelheide K, Catlin N, Gordon CJ, Morata T, Selgrade MK, et al. 2014. Cumulative risk: toxicity and interactions of physical and chemical stressors. Toxicological Sciences. 137:3-11. Rotroff DM, Wetmore BA, Dix DJ, Ferguson SS, Clewell HJ, Houck KA, LeCluyse EL, Andersen ME, Judson RS, Smith CM, Sochaski MA. 2010. Incorporating human dosimetry and exposure into high-throughput in vitro toxicity screening. Toxicological Sciences. 117(2):348-58. Schulte PA, Pandalai S, Wulsin V, Chun H. 2012. Interaction of occupational and personal risk factors in workforce health and safety. American Journal of Public Health. 102:434-48. Sexton K, Linder SH. 2011. Cumulative risk assessment for combined health effects from chemical and nonchemical stressors. American Journal of Public Health. 101 Suppl 1:S81-8.

22

Sexton K. 2012. Cumulative risk assessment: an overview of methodological approaches for evaluating combined health effects from exposure to multiple environmental stressors. International Journal of Environmental Research and Public Health. 9:370-90. Shin HM, Ernstoff A, Arnot JA, Wetmore BA, Csiszar SA, Fantke P, Zhang X, McKone TE, Jolliet O, Bennett DH. 2015. Risk-based high-throughput chemical screening and prioritization using exposure models and in vitro bioactivity assays. Environmental Science & Technology. 49(11):6760-71. Sipes NS, Wambaugh JF, Pearce R, Auerbach SS, Wetmore BA, Hsieh JH, Shapiro AJ, Svoboda D, DeVito MJ, Ferguson SS. 2017. An intuitive approach for predicting potential human health risk with the Tox21 10k library. Environmental Science & Technology. 51(18):10786-96. von Stackelberg KE. 2013. Decision analytic strategies for integrating ecosystem services and risk assessment. Integrated Environmental Assessment and Management. 9(2):260-8. von Stackelberg K, Guzy E, Chu T and Claus Henn B. 2015. Exposure to mixtures of metals and neurodevelopmental outcomes: a multidisciplinary review using an adverse outcome pathway framework. Risk Analysis. 35(6):971-1016, DOI: 10.1111/risa.12425. Tan YM, Leonard JA, Edwards S, Teeguarden J, Paini A, Egeghy P. 2018a. Aggregate exposure pathways in support of risk assessment. Current Opinion in Toxicology. 9:8-13. Tan YM, Leonard JA, Edwards S, Teeguarden J, Egeghy P. 2018. Refining the aggregate exposure pathway. Environmental Science: Processes & Impacts. 20(3):428-36. Teeguarden JG, Tan YM, Edwards SW, Leonard JA, Anderson KA, Corley RA, Kile ML, Simonich SM, Stone D, Tanguay RL, Waters KM. 2016. Completing the link between exposure science and toxicology for improved environmental health decision making: The aggregate exposure pathway framework. Environmental Science & Technology. 50(9):4579-86. doi: 10.1021/acs.est.5b05311. Thomas RS, Black MB, Li L, Healy E, Chu TM, Bao W, Andersen ME, Wolfinger RD. 2012. A comprehensive statistical analysis of predicting in vivo hazard using high-throughput in vitro screening. Toxicological Sciences. 128(2):398-417. Vinken M. Omics-based input and output in the development and use of adverse outcome pathways. Current Opinion in Toxicology. 18:8-12. Vrijheid M, Slama R, Robinson O, Chatzi L, Coen M, van den Hazel P. 2014. The Human Early-Life Exposome (HELIX): Project Rationale and Design. Environmental Health Perspectives. 122:535-544.

23

US EPA, United States Environmental Protection Agency. 1994. Ecological Risk Assessment Issue Papers. Risk Assessment Forum, Washington, DC. EPA/630/R-94/009 (NTIS PB95224192). US EPA, United States Environmental Protection Agency. 2001a. Water Quality Criterion for the Protection of Human Health – Methylmercury. EPA-823-R-01-001. U.S. Environmental Protection Agency, Office of Science and Technology, Office of Water. Washington, DC. US EPA, United States Environmental Protection Agency. 2001b. General Principles for Performing Aggregate Exposure and Risk Assessments. U.S. Environmental Protection Agency. US EPA, United States Environmental Protection Agency. 2002 . Guidance on Cumulative Risk Assessment of Pesticide Chemicals That Have a Common Mechanism of Toxicity. U.S. Environmental Protection Agency: Washington, DC. US EPA, United States Environmental Protection Agency. 2003. Framework for Cumulative Risk Assessment. U.S. Environmental Protection Agency, National Center for Environmental Assessment, Washington, DC. US EPA, United States Environmental Protection Agency. 2007. Concepts, methods, and data sources for cumulative health risk assessment of multiple chemicals, exposures, and effects: a resource document. U.S. Environmental Protection Agency, National Center for Environmental Assessment, Cincinnati, Ohio. US EPA, United States Environmental Protection Agency. 2012. Biomonitoring–An Exposure Science Tool for Exposure and Risk Assessment. EPA/600/R-12/039. US EPA, ORD, National Exposure Research Laboratory. Research Triangle Park, NC. US EPA, United States Environmental Protection Agency. 2014. Next Generation Risk Assessment: Incorporation of Recent Advances in Molecular, Computational, and Systems Biology. Final. September. National Center for Environmental Assessment, Office of Research and Development. EPA/600/R-14/004. US EPA, United States Environmental Protection Agency. 2016. Generic Ecological Assessment Endpoints (GEAEs) For Ecological Risk Assessment: Second Edition With Generic Ecosystem Services Endpoints Added. Risk Assessment Forum. EPA/100/F15/005. Wambaugh JF, Hughes MF, Ring CL, MacMillan DK, Ford J, Fennell TR, Black SR, Snyder RW, Sipes NS, Wetmore BA, Westerhout J. 2018. Evaluating in vitro-in vivo extrapolation of toxicokinetics. Toxicological Sciences. 163(1):152-69. Wild CP. 2005. Complementing the genome with an ‘‘exposome’’: the outstanding challenge of environmental exposure measurement in molecular epidemiology. Cancer Epidemiology, Biomarkers & Prevention. 14:1847–1850.

24

Wild CP. 2011. The exposome: from concept to utility. International Journal of Epidemiology. 41:24–32. Williams PR, Dotson GS, Maier A. 2012. Cumulative Risk Assessment (CRA): transforming the way we assess health risks. Environmental Science & Technology. 46(20):10868-74. Wittwehr C, Aladjov H, Ankley G, Byrne HJ, de Knecht J, Heinzle E, Klambauer G, Landesmann B, Luijten M, MacKay C, Maxwell G. 2017. How adverse outcome pathways can aid the development and use of computational prediction models for regulatory toxicology. Toxicological Sciences. 155(2):326-36. Zgheib E, Gao W, Limonciel A, Aladjov H, Yang H, Tebby C, Gayraud G, Jennings P, Sachana M, Beltman JB, Bois FY. 2019. Application of three approaches for quantitative AOP development to renal toxicity. Computational Toxicology. 11:1-3.