Are you concerned about the impact of PFAS Forever Chemicals on the environment?
Check out this valuable resource which delves into the latest research and findings on the effects of perfluoroalkyl and polyfluoroaklyl substances (PFAS) contamination and the methods for detection and remediation. Research into these chemicals continues to evolve, and strategies to address their presence worldwide continue to be developed. Get your free copy today and stay informed on this crucial topic.
Free article collection now available - click below
Free access
Sponsored by
Agilent
Wiley Analytical Science
Iti Check for updates
Environmental Toxicology and Chemistry—Volume 41, Number 6—pp. 1520-1539, 2022
Received: 20 September 2021   |  Revised: 25 October 2021   | Accepted: 28 February 2022 1 520
Hazard/Risk Assessment
The ECOTOXicology Knowledgebase: A Curated Database of Ecologically Relevant Toxicity Tests to Support Environmental Research and Risk Assessment
Jennifer H. Olker,3'* Colleen M. Elonen,3 Anne Pilli,b Arne Anderson,13 Brian Kinziger,b Stephen Erickson,b Michael Skopinski,b Anita Pomplun,b Carlie A. LaLone,3 Christine L. Russorn,3'1 and Dale Hoff3
aOffice of Research and Development, Center for Computational Toxicology and Exposure, Great Lakes Toxicology and Ecology Division, US Environmental Protection Agency, Duluth, Minnesota,
bGeneral Dynamics Information Technology, Duluth, Minnesota, USA
Abstract: The need for assembled existing and new toxicity data has accelerated as the amount of chemicals introduced into commerce continues to grow and regulatory mandates require safety assessments for a greater number of chemicals. To address this evolving need, the ECOTOXicology Knowledgebase (ECOTOX) was developed starting in the 1980s and is currently the world's largest compilation of curated ecotoxicity data, providing support for assessments of chemical safety and ecological research through systematic and transparent literature review procedures. The recently released version of ECOTOX (Ver 5, www.epa.gov/ecotox) provides single-chemical ecotoxicity data for over 12,000 chemicals and ecological species with over one million test results from over 50,000 references. Presented is an overview of ECOTOX, detailing the literature review and data curation processes within the context of current systematic review practices and discussing how recent updates improve the accessibility and reusability of data to support the assessment, management, and research of environmental chemicals. Relevant and acceptable toxicity results are identified from studies in the scientific literature, with pertinent methodological details and results extracted following well-established controlled vocabularies and newly extracted toxicity data added quarterly to the public website. Release of ECOTOX, Ver 5, included an entirely redesigned user interface with enhanced data queries and retrieval options, visualizations to aid in data exploration, customizable outputs for export and use in external applications, and interoperability with chemical and toxicity databases and tools. This is a reliable source of curated ecological toxicity data for chemical assessments and research and continues to evolve with accessible and transparent state-of-the-art practices in literature data curation and increased interoperability to other relevant resources. Environ Toxicol Chem 2022;41:1520-1539. © 2022 SET AC. This article has been contributed to by US Government employees and their work is in the public domain in the USA.
Keywords: Ecotoxicology; Environmental toxicology; Database; Systematic review; New approach methodologies; Controlled vocabulary; Interoperability; Chemical assessment
INTRODUCTION
Rapid and cost-effective methods are needed to evaluate chemicals for risk to human health and the environment. Regulatory agencies worldwide have been increasingly mandated to assess hazards of large numbers of chemicals while researchers seek to further existing bodies of knowledge. Typically, the starting point for hazard assessment and research of
This article contains online-only Supporting Information. 1 Retired.
* Address correspondence to olker.jennifer@epa.gov Published online 3 March 2022 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/etc.5324
contaminants is gathering existing empirical toxicity data. However, there are challenges in identifying, reviewing, and compiling toxicity test data in a thorough and transparent, yet efficient, manner. This is not a new problem, and the US Environmental Protection Agency (USEPA) began development of the ECOTOXicology Knowledgebase (ECOTOX) in the early 1980s with several ecosystem/taxa-specific databases to provide regulatory USEPA program offices with rapid access to ecotoxicity data.
Furthermore, there was a need for documented systematic and transparent literature searching, acquisition, and curation procedures to provide toxicological data for regulators and researchers across the USEPA and other federal and state agencies. Risk assessors needed a cost-effective means of
© 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxicological test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1521
locating relevant ecological effects data to use in chemical risk characterizations at hazardous waste sites and assisting in the assessment of potential hazards of chemicals while administering legislated mandates associated with the Federal Insecticide, Fungicide, and Rodenticide Act; the Clean Water Act; the Comprehensive Environmental Response, Compensation, and Liability Act/Superfund Amendments and Reauthorization Act; and the Toxic Substances Control Act. Thus, ECOTOX continued to develop and is now an authoritative source of curated, consistently updated, single-chemical environmental toxicity data relevant for ecological species, including both aquatic and terrestrial organisms, for use in risk assessment, risk management, and research.
Both toxicity testing and chemical regulation have evolved since the origination of ECOTOX. As the number of compounds and the need for rapid evaluation continue to increase, many new strategies have been developed to meet the demand for increased rate and efficiency in prioritizing, assessing, and predicting hazard and toxicity for both human health and ecologically relevant species. A National Research Council report, Toxicity Testing in the 21st Century: A Vision and a Strategy (2007), outlined a strategy to increase efficiency in evaluation of potential adverse effects of chemical exposure through a shift to greater reliance on high-throughput in vitro assays and computational modeling. This blueprint for change in toxicology contributed to recommendations for future toxicological risk assessments (Krewski et al., 2014; USEPA, 2014). Progress subsequent to the release of this National Research Council report includes advancements in new approach methodologies and technologies, including in vitro high-throughput assays, toxicogenomics, and in silico, structure-activity, and other computational modeling (Krewski et al., 2020; Thomas et al., 2019). While human health assessments have been the driver of these shifts, many of these concepts and methods are relevant and increasingly being applied to ecological assessments (Villeneuve et al., 2019). Recent mandates to reduce animal usage for toxicity testing have further focused efforts on new approach methodologies and led to efforts across the globe to transition to alternatives (European Chemicals Agency, 2017; Herrmann et al., 2019; Interagency Coordinating Committee on the Validation of Alternative Methods, 2018; USEPA, 2020a).
In this shifting toxicity testing paradigm, existing empirical data from in vivo toxicity testing have important roles in decision support on environmental effects of chemicals. At present, in vivo models continue to be the standard for chemical risk assessments. In addition, in vivo toxicity testing provides necessary empirical data for development of computational models and, moving forward, will bridge assessments based on new approach methodologies to those using the more traditionally defined apical endpoints. Verification and validation of new approach methodologies have been and continue to be major challenges, and most evaluation frameworks require biologically relevant in vivo data for comparison (see Parish et al., 2020). Thus, the identification and curation of toxicity data support the development, evaluation, and adoption of new approach methodologies by providing accessible existing data
for use in evaluations and modeling and identification of data gaps to target with additional testing.
Concurrent with the shifts in testing approaches to support regulatory toxicology, the use of transparent processes for the evaluation of existing toxicity data for use in risk assessment has evolved significantly in the environmental health field over the last two decades with the emergence of evidence-based toxicology, establishment of study evaluation methods, and adoption of systematic review approaches (Guzelian et al., 2005; Hoffmann & Härtung, 2006; Klimisch etal., 1997; C. Moermond et al., 2017; Rüden et al., 2017; Stephens et al., 2013, 2016, 2018; Thayer et al., 2014; Wikoff & Miller, 2018). The systematic review approach is a framework for identifying, evaluating, and synthesizing evidence that enhances transparency, objectivity, and consistency in reviews with a set of standard steps and clear criteria (Hoffmann et al., 2017; National Toxicology Program [NTP], 2019; Woodruff & Sutton, 2014). While multiple applications in the human environmental health field have made progress in incorporating aspects of systematic review (see USEPA, 2020b), ecotoxicological assessments have lagged in their adoption of systematic review methods. Through the decades, the ECOTOX team has developed operating procedures for identifying, reviewing, and extracting toxicity data from the literature which are consistent with the attributes of standardized guidelines for systematic reviews (NTP, 2019; Texas Commission on Environmental Quality, 2017) and systematic evidence maps (Wolffe et al., 2019).
To date, however, the ECOTOX project and literature review/data curation pipeline has not been described in the peer-reviewed literature and has not previously been framed within the context of current systematic review lexicon. The present study describes the recently updated ECOTOX (Ver 5), including an overview of the systematic methods used for literature search, review, data curation, description of the new user interface, summary of the type and extent of currently available data, and examples of applications and tools using data from ECOTOX. With the recent updates, ECOTOX strives to meet the needs of researchers and risk assessors with accurate, updated, and easily accessible information following the FAIR principles of "findable, accessible, interoperable, and reusable" data (Wilkinson et al., 2016). Although this is an ongoing process, ECOTOX Ver 5 represents a significant advancement in transparency and data accessibility, as well as advancements for data reusability and interoperability with other tools and databases. Soon after the release of the redesigned user interface, ECOTOX passed a major milestone and now includes over one million records of curated toxicity data from over 50,000 references. This evergreen database continues to support USEPA research and regulatory activities and is also a substantial source of toxicity records for other applications and databasing efforts including those for toxicity modeling (e.g., quantitative structure-activity relationships [QSARs], bio-accumulation), toxicity predictions (e.g., predicted no-effect concentrations and threshold values), and species sensitivity distributions (SSDs). Thus, this is an opportune time to provide the first comprehensive description of this project which has a long history of using systematic methods to identify and curate
wileyonlinelibrary.com/ETC
© 2022 SETAC
1522
Environmental Toxicology and Chemistry, 2022;41:1520-1539—J.H. Olker et al.
ecological toxicity data, with many similarities to contemporary systematic review methods, and recently added processes to increase transparency and consistency with FAIR principles.
METHODS
Over the decades, the ECOTOX team has developed a literature search, review, and data curation pipeline to identify and provide ecological toxicity data with consistency and transparency. We provide an overview of the systematic methods and procedures from chemical verification and development of search terms through literature review and data extraction. The following section (ECOTOX literature search, review, and data curation pipeline) summarizes the steps in the pipeline, and Figure 1 shows the flowchart of study selection, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines from Moher et al. (2009). Each step follows well-established standard operating procedures (SOPs), which are summarized in the present study. There are currently three SOPs that cover the steps of the pipeline (literature searches, citation identification, and applicability criteria; data abstraction; data maintenance), as well as SOPs for species verification and entry, chemical verification and entry, outreach and technical support, and software development, all of which are available on request. These SOPs
include project- and application-specific documentation, such as the details needed for reviewers to select and navigate through the appropriate interfaces and forms for screening references and extracting data, and are updated quarterly to reflect any changes in procedures, the incorporation of new efficiencies and shifts in file paths, and so forth.
Ecologically relevant toxicity studies for chemicals of interest are identified through comprehensive searches and review of the open and gray literature (note that the gray literature includes government documents and reports that are not typically included in databases which index peer-reviewed publications). Screening of references includes review of titles and abstracts followed by full-text review. References move forward in the pipeline following the established criteria for applicability (e.g., in terms of ecologically relevant species, chemical of interest, biological species identified, exposure concentration, and duration reported) and acceptability (documented controls and reported endpoints). From each study meeting these criteria (listed in Table 1), relevant details on chemical(s), species, study design, test conditions, and toxicity results are extracted into the structured database (see example data fields in Table 2 and full list in Supporting Information, Table S1). In the following section, we summarize this pipeline and the ECOTOX SOPs.
Many of the steps in this pipeline are consistent with systematic review methods which have recently been adopted
.a
'OD
Chemical verification and development of search terms
Conduct literature searches
Identify and acquire potentially applicable studies
Review literature for applicability to ECOTOX
Extract data and encode into ECOTOX Knowledgebase
Chemical-based search terms: Chemical name and CASRN, synonyms, tradenames, and other relevant forms such as metabolites, degradates, parent compound, and related chemicals of interest (when requested). Identified from multiple sources, including but not limited to STN, Pesticide Action Network (PAN), EPA's Pesticide Fate Database (PFATE), U.S. EPA's Chemicals Dashboard.
1
Literature search: Use chemical-specific search terms to query multiple literature search engines
Citations from:     ProQuest    Science Direct    ToxNet   Dissertation Abstracts Agricola/PubAg  Web of Science Already in Unify*
"Internal USEPA ECOTOX database
T
citations downloaded
Initial removal of duplicates
Title and abstract screening
n =_references
Not applicable (excluded): n -
references
Chem methods:_refs
Human health:_refs
Fate:_refs
Survey:_refs
Bacteria:_refs
No toxicant:_refs
Review:_refs
Modeling:_refs
ncident:_refs
Other:_refs
For review (full text screening)
n =_references
Chemical(s) of concern not in reference: n =_references
Did not meet acceptability criteria (excluded): n =_references
Data extracted from acceptable papers
n =_references with_total records
Aquatic:_references with_records
Terrestrial:_references with_records
Awaiting review and data extraction
Applicable, not yet reviewed due to funding, priorities, timing, etc. n = references
FIGURE 1: ECOTOXicology Knowledgebase (ECOTOX) literature search and study selection flow diagram, with general steps on left from PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses [Moher et al., 2009], dark gray) and ECOTOX pipeline (light gray). Each step is described in the section ECOTOX literature search, review, and data curation pipeline. CASRN = Chemical Abstracts Service registry number; STN = Scientific and Technical information Network; EPA/U.S. EPA= US Environmental Protection Agency.
© 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxicological test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1523
in toxicology and environmental assessments. However, ECOTOX is an ongoing project that continually identifies and reviews toxicity studies, and as such, the protocols were developed with a generalized approach compared to a systematic review for a single chemical or set of chemicals. In addition, ECOTOX provides ecologically relevant toxicity results to multiple partners and applications, and thus extracts and manages data suitable for predecisional analysis including sufficient chemical, methodological, and toxicity result details for users (e.g., USEPA program offices) to complete study quality evaluations and use the results to reach conclusions.
The procedures described in the present study have been reviewed and refined based on results from many years of identification of applicable references. The protocols are reviewed and updated on a continual basis, with the steps, sources, exclusionary reasons, and controlled vocabulary updated as needed. Updates and changes to the SOPs are made annually for quality assurance review and tracking, with relevant changes to the processes added to the documentation in both the User Guide and online "Help."
The ECOTOX Knowledgebase was developed to enter, review, and manage the references and toxicity results identified through the above-described pipeline. The Knowledgebase consists of two components: (1) the internal back-end Oracle database and data entry user interface referred to as "Unify," into which the chemical, reference, and toxicity tests and results are entered and managed (see section ECOTOX Infrastructure: Unify), and (2) the public Oracle database and web-based user interface that pulls information populated from Unify to provide data from studies that met the criteria for inclusion, which is generally recognized by the USEPA and our users as "ECOTOX" and recently underwent a complete redesign with added functionality and features (see section ECOTOX Ver 5.0: Updated web-based application). "Data" within ECOTOX are considered the records that are extracted from ecologically relevant toxicological studies, entered into Unify, and then provided in the public ECOTOX user interface. These records (or "data") include details on the chemicals, species, study design, test conditions, toxicity results, and statistically calculated or derived endpoints.
ECOTOX literature search, review, and data curation pipeline
Study selection criteria. The inclusion criteria for ECOTOX were developed to select ecological toxicity data possibly relevant for ecological risk assessments and subsequent risk-management decisions or policy. These inclusion criteria are summarized in Table 1 (second column) and consist of requirements for applicability as well as acceptability. The minimum data requirements for applicability include ecotox-icologically relevant criteria (e.g., ecologically relevant species, quantified chemical exposure amount and duration, biological adverse or neutral response to chemical exposure) as well as basic study reporting and quality requirements such as verifiable chemical and biological species (full descriptions in Supporting lnformation,Table S2). Wild species have been
prioritized for inclusion; however, test results for terrestrial domestic and laboratory species are used to fill data gaps when needed. In addition, acceptability criteria were developed to require documented controls and reported endpoints. These inclusion criteria can be stated in the form of a population, exposure, comparator, and outcome (PECO) statement commonly used in systematic review. The two formats are displayed side by side in Table 1.
Development of search terms. Most of the details in the inclusion criteria (and PECO statement) are general across all efforts; however, additional information is needed about each requested chemical to complete the exposure (E) section and develop search terms. Each chemical of interest is reviewed through a verification process to unequivocally link the chemical name to a specific Chemical Abstracts Service registry number (CASRN) and identify any synonyms, tradenames, and other relevant forms of the compound which should be considered for inclusion in the literature search (e.g., metabolites, degradants, parent compounds, related chemicals). This information is identified with resources, including, but not limited to, Pesticide Action Network, the USEPA's Pesticide Fate Database, and the USEPA's CompTox Chemicals Dashboard. The resulting chemical search terms are reviewed by the curation team and the partners requesting toxicity data for specificity to the chemical of interest and compiled for inclusion in the literature search.
Literature searches and sources of toxicity data. Literature searches are conducted using selected search terms as described above and shown in Figure 1. The majority of the literature searches are chemical-specific using the names and CASRNs for the target and related chemicals to search multiple databases within a narrow window of days to identify relevant references. The current list of databases includes Scopus, ProQuest, Dissertation Abstractions, PubAg, and Web of Science. This list of databases has remained fairly consistent over the years, although there have been recent shifts based on availability and use of web services (e.g., Scopus was recently incorporated as a replacement for ScienceDirect), and there are unique efforts that require additional sources (e.g., inclusion of PubMed when human health data are also of interest).
Chemicals included in these searches are based on the needs of our partners, such as requests from USEPA program offices and researchers within the Office of Research and Development, with anywhere from 48 to 90 chemical-specific searches conducted per year. Each literature search is conducted by ECOTOX staff specializing in this area, utilizing automation wherever possible and documenting the search terms and sources searched, the resultant number of citations from each search, and the disposition (or status) of each reference (applicable/not applicable and reason for exclusion, as identified in the next step) in standardized memos and file formats. This documentation is retained and provided to our partners for requested chemicals.
In addition to chemical-specific literature searches spanning all years using the databases described, titles and abstracts are
wileyonlinelibrary.com/ETC
© 2022 SETAC
1524
TABLE 1: Criteria for inclusion in the ECOTOXicology Knowledgebase (ECOTOX) shown side by side with these same requirements formatted as a population-exposure-comparator-outcome (PECO) statement
Requirements/inclusion criteria from ECOTOX standard operating procedure
General ECOTOX PECO
P (population)
E (exposure)
C (comparator/ control)
O (outcome)
Publication/data format
Ecologically relevant species Live, whole organisms
Organism taxonomic information verifiable against standard taxonomic sources Priority species are wild (test results for terrestrial domestic and laboratory species are used to fill data gaps when needed)
In vitro studies (with viable cells or tissue) flagged for possible inclusion as requested by programs NOT: humans, monkeys, bacteria, viruses, yeast
Verifiable Chemical Abstracts Service (CAS) number
Single-chemical exposure
Relevant to environmental exposure
Report exposure concentration, dose, or
application rate
Report duration of exposure
Sediment studies must have a water concentration reported to be included
NOT: Air pollution studies related to CO2 and ozone
Must have a control treatment3
• Biological effect measured
• Effect concurrent with associated chemical exposure
• Primary source of the data
• Study must be a full article in English NOT: Review article or abstract citation only
Animal: Aquatic and terrestrial species (live, whole organism) of any life stage (including preconception, in utero, lactation, peripubertal, and adult stages). Include vertebrates (wild mammals [e.g. Peromyscus sp.], fish, amphibians, reptiles, birds) and invertebrates (crustaceans, mollusks, insects, spiders, worms, other invertebrates). Bacteria and viruses are not included.
Plants: Aquatic and terrestrial species (live), all plants including algal, moss, lichen, and fungi species
Relevant forms:
Chemical of concern, name and CAS registry number (plus synonyms, tradenames); when requested: metabolites, degradates, parent compound, and related chemicals
Animal: Any exposure to relevant forms of the chemical of concern including via water, injection, diet, and dermal, with reported concentration and duration. Inhalation studies are excluded unless this is the primary route of environmental exposure (e.g., for volatile compounds).
Plants: Exposure to relevant forms of the chemical of concern via water or soil, with reported concentration and duration
A concurrent control group exposed to vehicle-only treatment and/or untreated control (control could be a baseline measurement)
All biological effects (including bioaccumulation from laboratory studies with concurrently measured water and tissue concentrations)
aStudieswith documented controls and statistically derived endpoints meet the acceptability criteria; however, there are instances in which data are included in ECOTOX without meeting these criteria, under the condition that they occurred in a publication with statistically analyzed endpoints compared to a control. For example, when standard methodologies were cited in the experimental design without explicitly naming a control exposure, median lethal concentrations (LC50s) were reported without a control exposure, or histopathological observations were reported without calculated endpoints or statistical analysis.
collected and reviewed monthly from the table of contents of several journals which historically have had a high frequency of ECOTOX-relevant references. This set of journals was identified based on evaluation of the applicability rate in the results from ECOTOX literature searches and is reviewed annually, with journals added or removed as needed. References identified with this literature search process are also reviewed with the inclusion criteria in Table 1 and often include studies for chemicals that are not currently prioritized by the USEPA's research or program offices. These references are added to the extraction queue, where they remain until either chemicals are prioritized or resources are available to complete the screening and data extraction process. Thus, curation includes toxicity data for both target chemicals for the USEPA's research and program offices as well as those of interest in the broader ecotoxicological community based on studies in recent publications.
In addition to references identified through the literature search process, ECOTOX includes data from several highly relevant toxicity databases. These include the USEPA fathead minnow acute toxicity database (Duluth Laboratory), the USEPA Office of Pesticide Programs Ecotoxicity Database, and
the US Geological Survey (USGS) Acute Toxicity Database, as well as independently compiled data sets from France, Germany, The Netherlands, and Russia (see User Guide, Appendix F, for details). These sources make up a small fraction (<6%) of the total records.
Screening references. The ECOTOX Knowledgebase has a tiered system for identifying studies to include (shown in Figure 1). Potentially applicable studies are first identified through a screening of the titles and abstracts from the literature search using the criteria (or PECO) described in Table 1. Studies that pass the title/abstract screening are acquired, and the full text of the article is reviewed using the same criteria. References which are excluded at either the title/abstract or full-text review stage are labeled ("tagged") with the primary reason for exclusion (e.g., "chemical methods," "human health," "survey," "mixture"; see Supporting Information,Tables S2 and S3 for minimum data requirements and a list of exclusionary reasons). The disposition (or status) of each reference is documented (applicable/not applicable, reason for exclusion) in the citation files and in Unify. Therefore, studies identified currently as
' 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxicological test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1525
"nonapplicable" are documented, and the citations are retained for potential future review for different projects or if there are changes in applicability or acceptability criteria.
Data extraction. For references that meet ECOTOX inclusion criteria, data are extracted for each reported toxicity test, including details on the test methods, species, chemical(s), and toxicity results (see examples in Table 2, full list in Supporting Information, Table S1). Toxicity tests within a reference are considered unique tests based on differing study designs (e.g., new chemical, new species, new life stage, new exposure type) and are given a computer-generated identification number ("test number"). Each toxicity test typically includes measurements for multiple types of effects, each with one endpoint or associated endpoints (statistical quantification or calculation of the observed effect, e.g., lethal concentration to 50% of test organisms [LC50], no-observed-effect concentration [NOEC], lowest-observed-effect concentration [LOEC]). The combined unique experimental design, effect measurement, and associated endpoint(s) are extracted as separate data records, with a unique "result number" assigned to each record. Thus, each reference may include multiple toxicity tests and multiple data records (see examples in Figure 2).
For each toxicity test within a reference, up to 90 different study entities (e.g., organism, chemical, media type, type of control, endpoint, statistical significance) are extracted in standardized fields with controlled vocabularies. To fully capture the relevant and reported information, many entities include additional fields for specific comments as well as ranges and units, resulting in possible totals of up to 300-400 data fields for terrestrial and aquatic studies, respectively. Extraction of study details and results is based on what is reported by the author(s) and follows SOPs using controlled vocabulary in a form-based data curator interface within the Unify system, which includes standard quality review for all extracted data. Supporting Information, Table S1 includes the list of data fields and associated appendices with controlled vocabularies (where relevant). The data fields and controlled vocabularies with definitions, are also documented in the online "Help" in "Data Field Definitions and Terms" (https://cfpub.epa.gov/ecotox/help. cfm?sub=wi-definitions) and the "ECOTOX Terms Appendix" (https://cfpub.epa. gov/ecotox/help.cfm?sub=term-appendix).
The established data fields include standardized unique identifiers (e.g., CASRN for chemicals, the National Center for Biotechnology Information's [NCBI] taxonomy identifier [TaxlD] for species) and ECOTOX-specific controlled vocabularies
TABLE 2: Types of data extracted from each reference for curation in the ECOTOXicology Knowledgebase (ECOTOX; if applicable and reported), with category, data fields, and examples of how specific fields in ECOTOX can inform study evaluation questions
Category
Data fields (not all inclusive)
Select study evaluation questions with relevant ECOTOX field(s)
Chemical
Species
Study design/test conditions
Test results
Chemical name, Chemical Abstracts Service registry number, grade, purity, formulation, carrier
Test-specific: Analysis, application type and
rate/frequency, number of doses, doses, concentration type (e.g., active ingredient or formulation), concentration/ dose associated with each effect and/or endpoint
Scientific and common name, taxonomy, life stage, age, initial and final weights, gender, source
Test method, media type, test location, exposure and study duration, control, experimental design, physical and chemical soil and water parameters
Effect (observation of a response): general effect groups and specific effect measurements, endpoint (quantification of an observed effect, e.g., LC50), trend, response site, effect percentage, statistical significance and level, observed duration (exposure duration when result observed), bioconcentration (BCF or BAF) with units
Is test substance identified?3,b,c Required for inclusion in ECOTOX
Is the purity of test substance reported?3,b,c Chemical Purity
Were chemical concentrations verified?0 Chemical Analysis (e.g., nominal vs. measured concentrations)
Is the species given?3,b,c Verifiable species {Scientific Name, etc.) required for inclusion in ECOTOX
Are the organisms well described?3,b,c Organism source, life stage, age, gender, initial and final weights
Are appropriate controls performed?a,b,c A control is required for inclusion in ECOTOX, type described in Control
Is a guideline method (e.g., OECD) used?a,c Test Method Are the experimental conditions appropriate and acceptable for the test substance and organism?a,b,c Test Method, Media Type, Test Location, Experimental Design, Physical and Chemical Soil and Water Parameters (e.g., pH, temperature, dissolved oxygen)
Are the reported effects and endpoints appropriate for the purpose, test substance, and organism?3 Effect Measurement, Endpoint
Is the response/effect statistically significant?3 Statistical Significance, Significance Level
The full list of data fields with definitions is in Supporting Information Table S1.
aCriteria for reporting and evaluation ecotoxicity data—relevance and reliability (Moermond et al., 2016). bRelevance and reliability of experimental data in human health risk assessment (Kaltenhäuser et al., 201 7). ^oxRTool, reliability assessment of in vivo toxicity studies (Schneider et al., 2009).
OECD — Organisation for Economic Co-operation and Development; LC50 —lethal concentration to 50% of test organisms; BCF-bioaccumulation factor.
bioconcentration factor; BAF -
wileyonlinelibrary.com/ETC
© 2022 SETAC
1526
(A)
Species
Effect group
Effect measurement
(B)
Endpoint Reference
Species
Effect group
* Measured at multiple exposure duratio Mortality, Development: 1, 2, 3,4 days Growth: 8 days Genetics: 5, 8 days
s, not all resulted in NOEC ä LOEC.
Effect measurement
Endpoint
NOEC
3 Measured at multiple pH: 5, 5.5, 6 EC50 at multiple pH, NOEC/LOEC at 5
LC50 LOEC NOEC
FIGURE 2: Examples of data extracted from references that have met applicability and acceptability criteria, with effects and endpoints measured and reported: (A) one study with amphibians and perfluorooctanesulfonic acid and (B) three studies with aquatic organisms and metals. Each reference could include multiple tests, each with a unique test design with different chemicals, species, life stages, exposure types, and so forth (e.g., Claesson & Tomqvist, 1988). From each reported test within a reference, each effect measurement and the associated endpoint(s) are extracted as separate data records in the ECOTOXicology Knowledgebase. Thus, there can be many data records for each reference. PFOS-K= perfluorooctanesulfonic acid-potassium; CRF = corticotropin releasing factor; HSP70 = heat shock protein 70; PPAR = peroxisome proliferator-activated receptor; NOEC = no-observed-effect concentration; LOEC = lowest-observed-effect concentration; NR = not reported; EC50 = effective concentration for 50% of test organisms; LC50 = lethal concentration to 50% of test organisms; LT50 = median lethal time.
developed through reviewing the ecotoxicological literature (e.g., media type, effect measurements, endpoints, test method and type). Numerical fields (e.g., organism age, number of doses, observed duration, concentration, temperature) include fields for the mean, minimum, and maximum values as well as operators (>, >=, <, =<, =) and units. Also, text "comment" fields are included for additional relevant study details. Toxicity results include reported endpoints (e.g., LC50, LOEC or lowest-observed-effect level [LOEL], NOEC or no-observed-effect level [NOEL], effective concentration forx% of test organisms [ECx]) for each effect measurement (e.g., total lipids, biomass, mortality) in a study, with measurements categorized into 23 effect groups (e.g., biochemistry, growth, reproduction, mortality, enzymes, genetics [which includes "omics" data]). Presently, there are 30 separate endpoints and over 5000 effect measurements defined in the ECOTOX controlled vocabularies. The ECOTOX Knowledgebase includes the treatment doses or concentrations associated with these effects and endpoints; however, because of priority given to statistically derived endpoints and limited resources, full extraction of responses at each treatment (or dose) is not completed. As described, each reference may include multiple toxicity tests, each with one or more effect measurements and endpoints, resulting in multiple records to represent all measured endpoints (including "no-effect" results). Examples of effect groups, effect measurements, and endpoints are shown
in Figure 2 for data extracted from individual studies and Figure 3 with data from multiple aquatic studies on aluminum chloride (CASRN 7446-70-0).
The data extracted into and managed within ECOTOX are useful for scoping or summarizing the extent or type of available data and providing relevant information for evaluating studies for inclusion in analyses, but no decisions are reached within ECOTOX. Data analysis and synthesis, as well as interpretation, are application- and assessment-specific and therefore are conducted outside of the structure of the ECOTOX Knowledgebase web application. However, the ECOTOX criteria for inclusion overlap with common study evaluation questions, and many of the data fields extracted from studies provide the necessary information to answer more detailed study evaluation questions (see Table 2). For example, the common study evaluation questions "Is the test substance unequivocally identified?" and "Is the test species given?" and "Were controls included?" have already been answered as "yes" for studies meeting the ECOTOX criteria for inclusion. Additional fields in ECOTOX that provide information for study evaluation include "test method," "doses," "exposure sample number," and "exposure duration," which could be used for evaluation of the experimental design either directly (e.g., "Is a guideline method used?" and "Is the number of animals per group given?" and "Is the exposure duration defined?") or to inform  professional judgments (e.g.,  "Were there an
i 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxicological test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1527
(A)
:						
		A			•      •_I	1 á
♦ ♦ ♦ ♦	♦		■	■ ■	■	I I A
♦ ♦ ♦ ♦ I  1  ♦ !	♦ • t	f ♦	♦ É T	T 1	1 ♦      T 1 m	á
¥     ¥     ♦ T t : * ♦ ♦ ♦	♦ • § ♦	t	;	t ♦	♦ " ♦ ■ é	A7 A-A
Invertebrates Fish Worms Algae
Amphibians Crustaceans Molluscs Insects/Spiders
Monoraphidium dybowskii LOEC: Biomass ECOREF #160584
Monoraphidium dybowskii NOEC: Abundance ECOREF #160584
Monoraphidium dybowskii NOEC: Biomass NOEC: Population growth rate ECOREF #160584
? vT
A/
(B)
■ ■
I ľ • •
0.01      • •
SS
Lecane quadridentata NOEC: Mortality ECOREF #162100
J* J
;sé-
NOEC LOEC NR
NR-ZERO
LC50
LT50
NR-LETH
EC50
BCF
IC10
MATC
IC25
EC50*
IC50
LC10
LC0
LC90
Lecane quadridentata LOEC: Mortality ECOREF #162100
FIGURE 3: Example data for aluminum chloride (Chemical Abstracts Service registry no. 7446-70-0) showing results for aquatic species with exposure concentration (y-axis) by type of effects (x-axis) and (A) species group (point color, shape) or (B) endpoint (point color, shape). Currently there are 731 aquatic records for this chemical from 102 publications, of which 492 have exposure concentrations that can be standardized to parts per million and displayed in the "Explore" data visualization plots. Note that darker shades indicate overlapping points. LOEC = lowest-observed-effect concentration; NOEC = no-observed-effect concentration; NR = not reported; NR-ZERO = 0% mortality; NR-LETH = 100% mortality; LCx = lethal concentration to x% of test organisms; LT50 = median lethal time; EC50 = effective concentration for 50% of test organisms; BCF = bioconcentration factor; ICx = x% inhibition concentration; MATC = maximum acceptable toxicant concentration.
appropriate number of doses?" and "Are test concentrations relevant in the context of addressed problem?" and "Is the duration of exposure appropriate for the studied endpoints?"). These fields can thus be informative for filtering studies for more restrictive inclusion criteria or to complement formal study quality or risk of bias evaluations that require review of the original publications. In such scenarios, we recommend that users of ECOTOX also consult the original publications because not all nuances of study design and toxicity results can be conveyed in the structured database. To aid in study evaluation, we recently released an optional output format which includes the fields most pertinent for evaluating relevance,
reliability, and/or risk of bias, based on recent review of evaluation guidelines (Kaltenhäuser et al., 2017; C. T. Moermond et al., 2016; Schneider et al., 2009).
Data maintenance and updates. As an evergreen database, ECOTOX continually adds references and records. In addition, previously added records are routinely updated to reflect current recommended terms and correct entry errors. For biological species, changes in taxonomy and scientific names are updated annually using a complete download from the USGS Integrated Taxonomic Information System (IT IS) database (http://www.itis.gov/). Species are updated with any changes in
wileyonlinelibrary.com/ETC
© 2022 SETAC
1528
Environmental Toxicology and Chemistry, 2022;41:1520-1539—J.H. Olker et al.
common names, synonyms, geographic terms, taxonomic serial number, and parent taxonomy. Previously recognized species names that are no longer used for taxonomic classification are retained as historical names and can be used in ECOTOX searches (with the currently accepted name provided in the output). Preferred names of chemicals, CASRNs, and terms in the controlled vocabulary (e.g., effect measurements, endpoints, exposure types) are also updated as needed to provide consistency and clarity. These updates are typically identified through quarterly review of terms or during incorporation of records from new literature and are documented in the relevant Unify modules and/or terms appendices. When deleted or alternate CASRNs are identified, these are maintained as historical CASRNs associated with the currently accepted/preferred CASRN, and all records are updated to the preferred CASRN. Corrections for entry errors are completed as needed. Entry errors are identified internally (through review and quality control [QC] procedures) as well as through reports by users. Any reported errors are reviewed in comparison to the original source, with corrections made and released with the quarterly updates. Documentation of the error and corrections is maintained within the project files.
ECOTOX infrastructure: Unify
The Unify system is the platform into which the chemical, reference, and toxicity tests and results are entered and managed. Unify maintains data provenance (all records traceable to the original source), enables quality assurance (QA) processes and QC checks, manages a backlog of potentially relevant references for future review and data extraction, and uses a unified extract-transform-load (ETL) processing platform for populating the ECOTOX public database reliably and efficiently. Unify was developed to modernize and collate the ECOTOX data entry process and initially merged the three separate taxa-specific databases (AQUIRE, PHYTOTOX, and TERRETOX) that previously stored all ECOTOX data. This change resulted in our current system, which is comprised of a sophisticated browser-based user interface using a ColdFusion model-view-controller framework on top of an enterprise-quality Oracle relational database with integrated modules for entry, management, and/or querying of chemicals, species, references, and/or toxicity data. The relational online transaction processing (OLTP) data model is optimized for multiuser real-time data entry and referential integrity.
Curation is conducted manually and standardized using a form-based graphic user interface for abstraction (locating and recording of details from published studies). Entry includes controlled vocabularies, drop-down and selection fields, and extensive data validation. The modules and vocabularies used are described as follows:
1. Chemicals: Chemicals are catalogued by the CASRN, with associated chemical name and synonyms from the CompTox Chemicals Dashboard (A. J. Williams et al., 2017; https:// comptox.epa.gov/dashboard/)   and   other   sources (e.g.,
PubChem, ChemSpider). Chemicals go through a process to verify chemical name, structure, and CASRN using these sources. Chemicals for which a CASRN cannot be located through these sources are labeled as "archived," and associated data are not entered until the chemical can be verified. The "Chemicals" module includes chemical properties, uses, and general structural groups (e.g., polyaromatic hydrocarbons [PAHs], neonicotinoids, polychlorinated biphenyls [PCBs]). As part of the updates described here, the Distributed Structure Searchable Toxicity Database Substance ID (DTXSID; Grulke et al., 2019) was added to increase interoperability with databases that include physicochemical properties, exposure, uses, and so forth.
2. Species: Test organisms are identified by their current scientific name, which is verified using IT IS. If the taxonomic data are not available with IT IS, other sources may be used (including taxa-specific sources such as AlgaeBase [www. algaebase.org] and Xenbase [www.xenbase.org]). For each entry in the "Species" module the verified name, taxonomic kingdom, nomenclature history, and verification sources are retained in the Unify record. Name synonyms (e.g., previous scientific names) and taxonomic/species groups (e.g., fish, standard test species) are included in the Unify "Species" module and available as ECOTOX query options. As part of the updates described, the NCBI TaxlD (www.ncbi.nlm.nih. gov/taxonomy; Federhen, 2012) was added and is available for approximately 80% of species in ECOTOX. The addition of the NCBI TaxIDs allows for greater interoperability with the USEPA Sequence Alignment to Predict Across Species Susceptibility (SeqAPASS) tool, which uses protein sequence comparisons to make predictions of chemical susceptibility across species (LaLone et al., 2016).
3. References: Each new reference that advances from title/ abstract screening (identified as applicable or needing full-text review) is added to the "References" module for further tracking and processing. This module includes standardized bibliographic information, workflow and tracking of the physical paper, full-text screening and abstraction information for references, and project report tracking and generation. When possible, digital object identifier (DOI), PubMedID, master record ID, and Health & Environmental Research Online ID are added to references in this module as additional identifiers for tracking and cross-referencing.
4. Toxicity: The "Toxicity" module was developed for efficient online data extraction from each reference, with separate sections for entity entry information of chemical(s), species, study design and parameters, and results. Entry parameters for all fields are defined (Supporting Information, Table S1) and selected from controlled vocabularies developed over the course of decades of reviewing ecotoxicological literature (terms are defined in the ECOTOX Terms Appendix; https://cfpub.epa. gov/ecotox/help.cfm?sub= term-appendix). Efforts to refine and map ECOTOX terms to standardized vocabularies (e.g., NCBI Entrez-Gene IDs, the National Library of Medicine's Unified Medical Language System) are underway.
© 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxicological test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1529
5. Administration: The "Administration" module includes internal lookup codes, users, and roles.
Intensive training and QA procedures are implemented throughout the steps in the ECOTOX pipeline, and trained abstractors follow the well-documented procedures described in multiple SOPs, with close interaction with a data coordinator. The QA process is an ongoing component of literature abstracting. Most data fields have validation files (lists of specific allowable terms) to control entries and electronic QA checks to ensure integrity of terms between data fields and consistency of data within fields. All entries are assessed for QA by trained staff to ensure that data have been accurately included and entered. More details on the data curation processes, including Unify modules, controlled vocabularies, and QA/QC processes, are available in the online "Help" section (https://cfpub.epa. gov/ecotox/help.cfm) and by request.
ECOTOX Ver 5: Updated web-based application
The curated toxicity data from studies which meet inclusion criteria are loaded from the Unify OLTP database into the public-facing ECOTOX data warehouse via a quarterly ETL process. The Oracle data warehouse is optimized for query operations by employing a star schema design and Oracle materialized views. The ECOTOX Ver 5 web-based application incorporates significant updates which were first implemented in 2018 to meet the data needs of current toxicological assessments, and updates are ongoing in response to increasing demands for rapid access to relevant and reliable data to meet regulatory mandates. These updates include integration of more diverse effect measurements (e.g., gene expression changes and biochemical responses), simplified outputs, and a completely redesigned user interface that incorporates data exploration and visualization tools which allow users to more efficiently search and use the existing toxicity effects data.
Redesign of the user interface was launched in August 2018 with the goal of increased accessibility and functionality for querying and exploring available data. New functionality and features have been added each quarter since our initial release. This change also improved the application architecture by separating the front-end user interface logic from the back-end database query logic. The web application utilizes an open-source development stack including HTML5, CSS, JavaScript, and JQuery/Ajax using the Node.js framework. In addition, the HighCharts JavaScript library was used for the new visualization plots feature.
In this newly designed user interface, data are presented and downloadable in tabular formats, with data query and filter options in the "Search" and "Explore" modules. Data can be queried directly through the "Search" module with options to input specific terms or filter by category (e.g., chemicals, effects, endpoints, species, test conditions, publication[s]). Data are available to download in Microsoft Excel or delimited (.txt) format in the "Search" module, with 22 fields included in the basic default display with the option to include additional data fields for download, if desired. The novel module "Explore"
allows discovery and exploration of available data through interactive filters for chemicals, species, effects, measurements, endpoints, and publication years. "Explore" also includes the option to visualize the data in dynamic plots, which display results by exposure concentration, duration, effect, chemical, species, and so forth. Selected query filters in "Explore" can be sent to the "Search" module for further refinement prior to downloading data. Details of how to navigate ECOTOX Search and Explore functions as well as query and download data can be found on the "Help" page (https://cfpub.epa.gov/ecotox/ help.cfm) and in the ECOTOX User Guide found therein, and described in the section Strategies for navigating ECOTOX Ver 5 for different applications.
RESULTS AND DISCUSSION
Current data landscape
The ECOTOX Knowledgebase captures a robust representation of the published ecotoxicological chemical testing literature for the past several decades. As such, the data can be utilized to reflect trends in the distribution of species, chemicals, and biological effect measurements in cumulative proportions and in shifts overtime in the field of ecotoxicology. In addition, reviewing the current data landscape can be used to prioritize literature searches and data curation, as well as identify data gaps for chemical and species groups. At the time of this writing (data update from June 15, 2021), ECOTOX includes extracted data from 52,274 references with toxicity data for 12,326 unique CASRNs and 13,610 biological species, for a total of 1,070,215 test records.
Chemicals with toxicity data in ECOTOX include, but are not limited to, groups such as metals and organometals, pesticides and herbicides, major ions, PAHs, polybrominated diphenyl ethers, PCBs, pharmaceuticals, and per- and polyfluoroalkyl substances. The number of references and records varies widely across chemicals (Supporting Information, Figures S1 and S2). Some chemicals have been well studied and documented in the published literature, such as copper sulfate (CASRN 7758-98-7), with 32,475 records extracted into ECOTOX from 2660 publications, while other chemicals are represented with only a few references (i.e., >6100 unique chemicals have only one reference in ECOTOX). The list of chemicals included in ECOTOX has been, and will continue to be, driven largely by USEPA regulatory and research needs; however, improved coverage across the chemical universe through emphasis on recent publications (with review of monthly publications from target journals) and prioritizing literature searches for data-poor chemicals is an objective moving forward.
The ECOTOX Knowledgebase includes data for >13,000 biological species, with a distribution that reflects the somewhat uneven taxonomic representation in the ecotoxicological literature. For example, a select set of model species (e.g., rainbow trout, northern bobwhite quail, Norway rat) has historically been used for most toxicity testing, and the 227 "standard test species" in ECOTOX (described under "Species Verification" in "Help," https://cfpub.epa.gov/ecotox/help.cfm?
wileyonlinelibrary.com/ETC
© 2022 SETAC
1530
Environmental Toxicology and Chemistry, 2022;41:1520-1539—J.H. Olker et al.
sub=wi-documentation) not surprisingly represent 45% of the records. In fact, 100 of the 13,610 species account for 50% of the total number of records, whereas >9000 species have data from only one or two references (Supporting Information, Figure S3). Fish are the most highly represented taxonomic group, with 25% of the records (273,693 records) and >13,000 references in ECOTOX. Taxonomic groups with the fewest records and references are reptiles, moss, and hornworts. Within the taxonomic groups, the amount of available data varies greatly across individual species. For example, looking at the fish species with the highest number of records in ECOTOX, four standard test species (rainbow trout [Oncorhynchus my-kiss], fathead minnow [Pimephales promelas], common carp [Cyprinus carpio], and bluegill [Lepomis macrochirus]) represent 0.4% (4/1000) of all fish species in ECOTOX yet account for 33% of both the references (4433/13,281) and records (90,183/273,693) for fish (Figure 4A). Zebrafish {Danio rerio) is among the top five fish species (based on number of records); however, prior to 2009, this species contributed relatively few records to the total number of records for fishes. Currently, zebrafish is ranked second in total number of records for fish species (Figure 4B), in part because of its "crossover" status as a human health model species for neurodevelopmental toxicity testing (d'Amora & Giordani, 2018). Zebrafish together with fathead minnow, rainbow trout, common carp, and bluegill reflect 47% (127,520/273,693) of records from fish species. This uneven distribution of available ecotoxicity data highlights the need to conduct "refresh" literature searches for chemicals to potentially fill data gaps for important species or species groups (e.g., threatened and endangered species), as well as to develop linkages to tools that evaluate species similarities (e.g., the USEPA's Web-based Interspecies Correlation Estimation [Web-ICE], SeqAPASS).
The types of effects studied and reported have changed substantially over the decades that ecotoxicological research has been conducted. Many publications now include a wider variety of types of effects (e.g., behavioral, transcriptomics) in
addition to the standard growth, development, reproduction, and mortality endpoints historically reported in the literature. The ECOTOX Knowledgebase continually incorporates effect measurement terms to the controlled vocabulary to reflect these shifts in reported toxicity results. Currently, ECOTOX includes toxicity data from 6070 author-reported unique effect measurements, including a great diversity of endpoints beyond the standard apical toxicity measurements (growth, reproduction, and mortality). These include measurements of organ- and cellular-level effects (e.g., histopathology, organ-to-body weight indices, red and white blood cells); hormone, enzyme, and other biochemical changes (e.g., thyroxine, cholesterol, aromatase activity); genetic effects (e.g., DNA or RNA concentration, DNA methylation, chromosomal breaks, transcription of a wide variety of genes); chemical accumulation (e.g., body burden, uptake); and behavioral changes (e.g., general activity, feeding behavior, swimming, orientation; Figure 5A). Measurements of mortality, growth, and reproduction make up 47% of the total records and are included in 39%, 32%, and 12% of references, respectively. Subapical biochemical, cellular, and genetic effects are included in 28%, 9%, and 7% of references, respectively; and combined these three types of effects account for 20% of the records.
The results in ECOTOX provide insight into historical trends of ecotoxicological testing. Data extracted from the literature through the decades show that growth and mortality have been, and continue to be, the most often reported effects. Studies published prior to 1980 primarily report effects on mortality, growth, and physiology, with some reproduction (<10% of references) and population (-20% of references, mostly for algae and plants) effects (Figure 5B). The percentages of references reporting growth and mortality have stayed consistent through time (36%-42%), while the numbers of studies reporting reproduction and behavior have grown from 10% to 15% and from 6% to 12%, respectively. With technological advances in analytical, biochemical, and behavioral measurement techniques, measurements in reported toxicity
(A)
Number of records
Other, 108,478
(B)
700
500
400
200
100
Rainbow trout ■ Bluegill Fathead minnow Common carp Zebrafish
1940s     1950s     1960s     1970s 1
2000s 2010s
Publication date
FIGURE 4: Distribution of data in ECOTOXicology Knowledgebase for fish species: (A) number of records across all fish species (as of June 2021 update); (B) number of references for five commonly studied fish species by decade of publication.
© 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxlcologlcal test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1531
48/° e.7% Publication date
FIGURE 5: Distribution of types of effects data in ECOTOXicology Knowledgebase (ECOTOX): (A) number of records by general type of effects (as of June 2021 update); (B) percent of references that include each general type of effect (effect groups as bars) by decade of publication with cumulative number of records in ECOTOX (black line).
testing have shifted. Increases in reported biochemical effects started in the 1980s, and references with biochemical end-points doubled from 18% in the 1960s and 1970s to nearly 40% in the 2000s. These changes were followed by increasingly more cellular and genetic effects starting in 2000, with references increasing from approximately 3% to >10%. While traditional apical endpoints (growth and mortality) continue to be included in ecotoxicological testing, technological advances, in combination with chemical regulatory policy directives to reduce animal usage and to identify suborganismal effect levels for apical endpoints in chemical assessment, have resulted in dramatic increases of biochemical, genetics, and cellular end-points reported over the last decade. This trend is likely to continue.
Applications and users of the ECOTOX
In addition to supporting USEPA research and regulatory activities, data curated into ECOTOX have been used for many applications in government, industry, and academia (Table 3). Within the USEPA, the data in ECOTOX contribute to development of ambient water quality criteria for aquatic life, ecological risk assessments for chemical registration and reregistration, chemical prioritization and assessment under the Toxic Substances Control Act, and site-specific assessments and emergency response. Based on those that contact the ECOTOX team, other users include several (US) federal agencies, states, and tribes; international entities; academic researchers and educators; and industry. Because no registration or log-in is required to search ECOTOX data, other resources such as Google Analytics and peer-reviewed publications have been relied on for information on the volume of use, types of users, and applications. Recent volume of use was assessed by Google Analytics from March 2019 to June 2021, during which time there was an average of 19,820 page views per month (range of -14,400 to -23,900 per month).
There are hundreds of publications that reference ECOTOX, with the majority occurring over the past two decades, from a variety of journals related to toxicology, supporting work conducted in numerous countries around the world (Supporting Information, Table S4). Many of these publications use toxicity data from ECOTOX for specific applications, while others, including many book chapters on ecotoxicology, refer to ECOTOX as an example of available toxicity data in general but also for specific compounds. Many tools and applications incorporate data from ECOTOX, including those focused on SSDs (e.g., the USEPA's Web-ICE, the National Oceanic and Atmospheric Administration's Chemical Aquatic Fate and Effects), predicted-no-effect concentrations and threshold values (e.g., ecological threshold of toxicological concern, EnviroTox, NORMAN), QSARs (e.g., the Ecological Structure-Activity Relationships predictive model, Toxicity Estimate Software Tool, the Organisation for Economic Co-operation and Development's [OECD's] QSAR Toolbox, Assessment Tools for the Evaluation of Risk), bioaccumulation modeling and validation, and adverse outcome pathway development (see Table 3 for examples). The recent updates to ECOTOX increase the ease of identifying and accessing relevant data for applications such as those listed and as described in the following section.
Strategies for navigating ECOTOX Ver 5 for different applications: Components and examples
This section provides an overview of how to navigate ECOTOX to query or download data, with a focus on examples of how the updates and improvements have increased the accessibility and transparency of ECOTOX data. Extensive details can be found on the ECOTOX "Help" page (https:// cfpub.epa.gov/ecotox/help.cfm) and in the User Guide (most recent version available online, link in "Help" page). Illustrative examples of applications of ECOTOX are briefly described in
wileyonlinelibrary.com/ETC
© 2022 SETAC
1532
Environmental Toxicology and Chemistry, 2022;41:1520-1539—J.H. Olker et al.
TABLE 3: Applications using empirical toxicity data curated in the ECOTOXicology Knowledgebase, with examples, tools, and references
Applications
Examples, tools, and references
Toxicity reference values and benchmarks
Species sensitivity distributions
Predicted-no-effect concentrations and threshold values
Modeling (e.g., QSARs, quantitative ion character-activity relationships, chemical toxicity distributions)
Bioaccumulation modeling and validation of in vitro extrapolation models
Linking environmental contaminants to potential effects
Comparison to new toxicity studies
Adverse outcome pathway development and weight of evidence
Inform study design
USEPA's eco-SSLs (https://www.epa.gov/chemical-research/interim-ecological-soil-screening-level-documents)
USEPA aquatic life ambient water quality criteria (https://www.epa.gov/wqc/ national-recommended-water-quality-criteria-aquatic-life-criteria-table) USEPA risk evaluations under Toxic Substances Control Act https://www.epa. gov/assessing-and-managing-chemicals-under-tsca/chemicals-undergoing-risk-evaluation-under-tsca
Sorensen et al. (2007), Ra et al. (2008), Nam et al. (2012), Brock and Van Wijngaarden (2012)
Example A (details in Supporting Information)
USEPA Endangered Species Act biological evaluations of conventional pesticides (https://www3.epa.gov/pesticides/nas/revised/revised-method-march2020.pdf)
USEPA's Web-ICE (https://www3.epa.gov/webice/)
NOAA's CAFE (https://response.restoration.noaa.gov/oil-and-chemical-spills/ chemical-spills/response-tools/cafe, html)
Bielmyer et al. (2006), Schuler et al. (2008), Liu et al. (2009), Yang et al. (2014), Demetrio et al. (2012), Dornfeld et al. (2019), Stadlinger et al. (2018) Example B (details in Supporting Information)
EnviroTox (https://envirotoxdatabase.org/)
NORMAN (https://www.norman-network.com/nds/ecotox/)
Rial et al. (2010), Connors et al. (2019), Dulio et al. (2020)
ECOSAR (Ecological Structure-Activity Relationships; predictive model,
https://www.epa.gov/tsca-screening-tools/ecological-structure-activity-
relationships-ecosar-predictive-model)
TEST (Toxicity Estimate Software Tool; https://www.epa.gov/chemical-
research/toxicity-estimation-software-tool-test)
OECD QSAR Toolbox (https://qsartoolbox.org/)
USEPA's ASTER (Assessment Tools for the Evaluation of Risk)
Ownby and Newman (2003), Voigt et al. (2004), Bermudez-Saldana et al.
(2005), Lee et al. (2006), Rosenbaum et al. (2008), Traore et al. (2018), E. S.
Williams et al. (2011)
Fay et al. (2014)
Bacey and Spurlock (2007), Johnson et al. (2008), Kostich et al. (2017), Tousova et al. (2018)
Example C (details in Supporting Information)
Dardenne et al. (2007), Ralston-Hooper et al. (2009), Fai and Grant (2010), Bettiol et al. (2016), Dornfeld et al. (2019)
Russom et al. (2014), Fay et al. (2017)
George and Liber (2007), Leboulanger et al. (2011), LeBlanc et al. (2012), Bezirci et al. (2012), Alexander et al. (2013)
USEPA—US Environmental Protection Agency; SSL —soil screening level; ICE — Interspecies Correlation Estimation; NOAA—National Oceanic and Atmospheric Administration; CAFE — Chemical Aquatic Fate and Effects; QSAR— quantitative structure-activity relationship; OECD — Organisation for Economic Co-operation and Development.
the section Examples of ECOTOX applications, with details and associated tables and figures provided in the Supporting Information (Tables S5-S9 and Figures S4-S10). Note that the original scientific papers should be consulted for awareness of any nuances of the data retrieved from ECOTOX in the context of decision-making.
Initial landing page and getting assistance. The initial landing page for ECOTOX is the "Home" page, which provides a general overview and contains links to the other pages within the application (e.g., "Search," "Explore," "Help"), as well as
information on ECOTOX data statistics (e.g., number of records for each update and as totals) and other databases. Important, yet often overlooked, are the links from the Home page to "Help" and "Contact Us." The "Help" page provides a wealth of information in general categories such as "Starting Out" (how to begin navigating and searching within ECOTOX, as well as frequently asked questions and links to the PDF version of the User Guide), "How do I ..." (details on how to perform searches, select query parameters, and format output), "What is..." (overview on database including species and chemical verification sources used by ECOTOX curators, details on data
© 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxlcologlcal test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1533
field definitions, and terms in ECOTOX's controlled vocabulary), and "More Resources" (glossary of terms and links to related websites). When additional information is needed or if users have suggestions to improve ECOTOX, comments and questions can be submitted through the "Contact Us" link.
Strategies to identify relevant data in ECOTOX. The
strategies available for users to identify relevant data are one of the most impactful changes from the previous versions of ECOTOX. Database retrievals can now be conducted using either the "Search" module to directly query specific species, chemicals, or effects with expanded options to customize many output fields or the "Explore" module to investigate and visualize the data available. The new "Search" module provides a direct method to retrieve data, comparable to the previous "Advanced Query" in ECOTOX, Ver 4, with expanded options to iteratively revise the query criteria, customize output to include specific data fields, and review the data in tabular format before downloading. The query parameters available in "Search" include options for chemicals (e.g., CASRN, chemical name), species (e.g., scientific name, taxonomic group), effects (e.g., type of effect, specific measurements), statistically derived indicators of effects/no-effects (endpoints such as LC50, EC50, NOEL, and LOEL), test conditions (e.g., study site type [laboratory, field], exposure media (freshwater, soil), route of chemical exposure [oral, diet]), and publication options (e.g., author name, publication year). Selections may be further filtered before continuing a search or updated after reviewing the report (tabular data) based on the initial parameters, and the output can be customized to include or exclude fields for viewing or exporting in Excel or delimited formats. Users can also view and download the list of references associated with records meeting query parameters.
Version 5 of ECOTOX includes the new module "Explore," which is especially helpful when search criteria are not known and/or if a visual representation of general and specific data trends is desired. This module includes a limited set of query filters and output formats, which focus on helping the user to investigate data availability through interactive queries and data visualizations. At any point, the query parameters in "Explore" can be sent to "Search" for further refinement with additional query parameters and review or export of more comprehensive information with additional data fields. The "Explore" module starts a user with predefined or custom groups for chemicals, species, or effects groups, with query filters for chemicals, species, effect groups, effect measurements, endpoints, and publication years. As parameters are selected, the other parameters and tables respond reactively, which allows the user to refine or expand parameters to identify the most relevant data for a specific question or goal. For example, once a chemical or chemical group is selected in "Explore by Chemical," the query filters will display the number of species, effect groups, effect measurements, endpoints, and so forth for which there are data in the Knowledgebase. Selection within the query filters is then limited to the categories with data. When any of the options within a query filter are selected, such as limiting the exploration
to only fish in "Species Group," the rest of the query filters will only show options for the chemical(s) selected for which there are data for fish species. This reactive feature in "Explore" allows the users to expand or restrict the query based on this immediate knowledge of data availability.
Data output in "Explore" are presented in three formats: "Group Summary" (summary of number of records and publications by chemical, species, or effect group), "Records" (resulting records summarized by effect and endpoint), and the "Plot View" (dot or scatterplot of records by exposure concentration or duration by effects, endpoints, species, chemicals, and so forth, including a simplified table of records below the plot). Data presented in each of the tables and plots can be further refined using the query filters. Plot View allows visualization of the data, which provides a general concept of the range of available data. The plots are interactive, with the option to "zoom" into the plot, deselecVselect items in the plot legend, and/or display a line at a specified concentration. There are some limitations to the data visualization feature (maximum display of 3000 records, only includes data that can be converted to exposure concentrations of parts per million). Therefore, it is recommended that the user sends the query parameters in "Explore" to the "Search" module before downloading data. Both the "Search" and "Explore" modules are designed to search on all data unless users restrict the search by choosing specific criteria of interest. Further details and examples for querying and exploring the available data are included in "Help" sections.
Data downloads. The ECOTOX Knowledgebase allows downloads of all data available on the public-facing website. Results from "Search" include the most comprehensive set of data fields and can be downloaded either as a Microsoft Excel spreadsheet or a delimited file format. The Excel spreadsheet includes all data records, the list of references from which the data were extracted, and the query parameters applied by the user in these output files for improved documentation and reproducibility of searches. When delimited files are preferred by the user, the data records, list of references, and search parameters are available to download as separate files from the "Search" module. Within the "Explore" module, comma separated files (.csv) can be downloaded for the group summary table, the records table, or the simplified table below the plot; however, these do not provide the same level of detail or the options for customizing output fields that are available in the "Search" module. In addition, users can download delimited ASCII files of the entire aquatic or terrestrial raw data. The ASCII file download does not require specific software, although it requires combining various files together to view entire records because the data are divided into two sections: aquatic and terrestrial. Within each of these sections, users will find data tables, field descriptions, and graphical relations of the data structure.
Examples of ECOTOX applications. The updated options for querying and downloading data greatly improve the end user's ability to efficiently retrieve targeted toxicity data, with
wileyonlinelibrary.com/ETC
© 2022 SETAC
1534
Environmental Toxicology and Chemistry, 2022;41:1520-1539—J.H. Olker et al.
added benefits of flexibility in the output formats, transparency and documentation of search results, and the embedded information to aid in linking to external databases and computational platforms. In this section, we present examples of how the new ECOTOX can be used to retrieve toxicity data for several common use cases, with brief descriptions (full details in the Supporting Information). Each example is intended only to demonstrate accessing toxicity results through "Search" and "Explore" for identification, filtering, and export of relevant data; further evaluation and analysis of downloaded data is conducted in other platforms. See Table 3 for published studies with similar applications as these examples.
1. Example A: Development of toxicity reference values. Toxicity data from ECOTOX can be used in the development of toxicity reference values (TRVs; see Table 3). In this example, we used "Explore" to determine the scope of toxicity data available for various cadmium (Cd) and biological species (Supporting Information, Table S5) and observed potential trends in relative potency among chemical species and to develop testable hypotheses (Supporting Information, Figure S4). This exploration of available data allowed us to determine that growth effects data in mammalian species could be used to evaluate how metal speciation affects TRVs. The "Search" module was then used to select and export data (Supporting Information, Table S6) specific to the testable hypothesis: Does metal speciation affect TRVs? After export from ECOTOX, the data records were further filtered based on methodological data fields (e.g., exposure method, number of doses, use of a control, and exposure duration), and dose-response modeling of growth effects (using the Toxicity Relation Analysis Program, Ver 3.1; https://archive.epa.gov/med/ med_archive_03/web/html/trap.html) showed that ECx values for cadmium chloride were 4-fold lower than for cadmium acetate (Supporting Information, Figure S5). Additional details for this example are in the Supporting Information (Supplemental for Example A).
2. Example B: Development of SSDs. Toxicity data from ECOTOX are routinely used in the development of SSDs (see Table 3). In this example, we used "Explore" and "Search" to identify empirical toxicity data for the development of a chlorpyrifos SSD for aquatic invertebrates. Through the recently added features in "Explore," we queried the aquatic data for chlorpyrifos (CASRN 2921-88-2); filtered these results to limit the species group, effect groups, effect measurements, and endpoints to those of interest; and reviewed the scope of available data by species, class, and general species group (Supporting Information, Figures S6 and S7). These query filters were sent to the "Search" module, where the data were exported with the data fields relevant for study evaluation and formatted for import into the SSD Toolbox (Etterson, 2020). After export from ECOTOX, the data records were filtered to include only 2- and 4-day studies and reviewed for inclusion in the SSD (Supporting Information, Table S7). The data records which were identified as
suitable for inclusion were imported into the SSD Toolbox to create a preliminary SSD that resulted in 5% hazardous concentration of 2.024 x10~5 using the normal distribution fit to the toxicity data with maximum likelihood (Supporting Information, Figure S8). Additional details for this example are in the Supporting Information (Supplemental for Example B). 3. Example C: Linking chemical detections and concentrations to effects data. Toxicity data in ECOTOX can provide relevance for potential effects of the chemicals detected in environmental samples. In this example, we used the "Explore" module to determine the scope of aquatic toxicity data available for 109 chemicals detected in surface water samples collected in the Great Lakes basin (Lee et al., 2015) using the chemical identifiers provided (CASRNs) and summarizing by the types of chemicals (Supporting Information, Table S8). Available data were further reviewed with the visualization options in "Explore" (see example in Supporting Information, Figure S9), and then, after additional filtering, query filters were sent to the "Search" module for exporting the data (Supporting Information, Table S9). Summaries were completed in Excel and R to identify data-poor compounds as well as compare measured concentrations with observed effect concentrations (Supporting Information, Figure S10). Additional details for this example are in the Supporting Information (Supplemental for Example C).
Challenges and limitations
As with any ongoing project, ECOTOX has recognized challenges and limitations. One of the largest limitations is the sheer immensity of the ecotoxicological literature; even with thousands of chemicals and species, data currently in ECOTOX are a subset of all of the compounds tested with ecologically relevant species. The identification of studies for inclusion is primarily through targeted chemical-specific literature searches based on the needs and requests from USEPA program offices or research teams (as described in the section Literature searches and sources of toxicity data). In an effort to address the issue of underrepresented chemicals and species, monthly review of relevant recent publications, prioritization of species groups with limited data, and the recent incorporation of data analytics tools to decrease screening time have been incorporated. These efforts have aided the prioritization of studies with chemicals and/or species that are currently underrepresented. However, there remains a backlog of >60,000 potentially relevant references in the Unify "References" module waiting to be prioritized for full review and data extraction. In addition, ECOTOX is limited to the details and results reported by study authors. This retains the intent of the authors and avoids potentially erroneous extrapolation or interpretation; however, this results in fields being left empty when not reported in a study. This issue of insufficient reporting is across toxicology, as well as other disciplines, and is currently being tackled by several journals and working groups (e.g., Society of Environmental Toxicology and Chemistry, 2019).
© 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxlcologlcal test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1535
In addition, relying on author-reported details has resulted in potentially duplicative or overlapping terms, in particular for the specific effect measurements and endpoints reported. The employment of controlled vocabularies has enabled use of consistent terminology by relating terms used by authors to the common set of verified terms, and the integration of ontologies will further aid this consistency.
Another major challenge is balancing the need to respond to the evolving toxicology testing and regulatory environment with the responsibility to maintain the current processes to curate toxicity results from traditional in vivo toxicity tests. Continuing the identification and incorporation of in vivo studies is essential for providing structured data for comparison and evaluation of alternative methods. As incorporation of new approaches expands, the recommended data reporting and evaluation criteria for relevant studies have started to become standardized (e.g., OECD, 2017, 2018; Roth et al., 2021). Already ECOTOX has begun incorporating in vitro studies and has added terms to the controlled vocabularies to reflect these methodologies (e.g., "exposure type"), with additional fields and terms expected to be added as guidance is developed. With both the current and future records, ongoing work is needed to update the previously curated data to meet current standards and use currently recommended terminology. This is not a new challenge and is already incorporated into the ECOTOX processes (see section Data maintenance and updates).
Interoperability and future directions
The updates made to the ECOTOX Knowledgebase increase the efficiency in identifying and retrieving relevant data, with added functionality to explore and visualize available ecological toxicity data, thus supporting the increasing need for rapid access to toxicity test results used for risk assessment and chemical decision-making. This ready access to a wealth of existing toxicity data along with the ongoing addition of data to the Knowledgebase will also help the USEPA efficiently review possible chemical effects on ecologically relevant species, identify data gaps, prioritize testing, and evaluate new approach methodologies to support the decision-making for the increasingly large universe of chemicals. Future applications and plans for improvements include increased functionality and interoperability to allow for integration with other USEPA databases, new text mining and data visualization programs, as well as support for new and evolving tools for predictive toxicology.
Throughout the recent updates to ECOTOX, there was a recognized need to incorporate support for data integration and interoperability. Typical applications that use toxicity test results from ECOTOX (e.g., ecological risk assessment, SSDs, site-specific hazard evaluations) incorporate additional toxicity, chemical, and biological data from other sources. In addition, computational toxicology and integration of new approach methodologies require the ability to link multiple disparate data sources (Watford, Edwards, et al., 2019). Thus, work is being conducted with developers of web portals for chemical information to incorporate linkages into ECOTOX, including the OECD eChemPortal (https://www.echemportal.
org/echemportal/, ECOTOX chemical list added in 2020) and the USEPA's CompTox Chemicals Dashboard (https://comptox.epa. gov/dashboard/chemical_lists/ECOTOX_V2, added in 2017).
In addition, information is being incorporated into ECOTOX which will enable the integration and exchange of data with public data resources having similar (e.g., toxicity data in Tox-RefDB; Watford, Pham, et al., 2019) and different (e.g., phys-icochemical properties in the CompTox Chemicals Dashboard, biological pathways in Gene Ontology and Kyoto Encyclopedia of Genes and Genomes) types of data following FAIR data principles (Wilkinson et al., 2016). Initial steps to increase the accessibility and ease of locating data within ECOTOX have included incorporation of unique identifiers for biological species (NCBI TaxIDs, added in 2018) and chemicals (DTXSID, added in 2020). Further work for data interoperability and information discovery includes incorporation of unique identifiers for genes and proteins (e.g., NCBI Entrez GenelD, UniProt IDs) and mapping ECOTOX controlled vocabularies to existing toxicology database vocabularies (e.g., Unified Medical Language System) and ontologies (e.g., Chemical Entities of Biological Interest). In combination, these unique identifiers and mappings will enable linking to and from a variety of chemical, toxicological, and biological databases; increasing the accessibility of data in ECOTOX; and supporting advanced querying capabilities. These additions will ultimately contribute to making ECOTOX machine-readable and computable, which in turn supports future goals to advance integration into applications such as adverse outcome pathway development and semantic phenotypic mapping (Ives et al., 2017; Wang et al., 2019; Whaley et al., 2020).
Complementary to the addition of new information for interoperability, new technologies in artificial intelligence, including machine learning and data analytics and software applications, are being incorporated to increase the efficiencies of the pipeline. In 2020, we began testing and utilizing applications for automated literature searching and machine learning and artificial intelligence techniques for title and abstract screening of references (e.g., AbstractSifter [Baker et al., 2017], SWIFT-Review [Howard et al., 2016], and SWIFT-ActiveScreener [Howard et al., 2020]). Preliminary results (not reported) have demonstrated that these tools provide time savings for most typical chemical searches and reviews conducted by the ECOTOX team. Efficiencies gained include increased speed at which ECOTOX personnel can review literature and extract data, which, along with enhanced ECOTOX interoperability with other data sources, will aid in meeting the current demand for rapid access to toxicity data for risk assessment and environmental research. As we look to the future, other challenges (e.g., complex mixtures, nanoparticles) are recognized, and possible additions and/or modifications to the ECOTOX Knowledgebase are being considered.
SUMMARY AND CONCLUSIONS
The ECOTOX Knowledgebase has long provided curated data extracted from ecologically relevant toxicity studies for use in  broad  applications in environmental  research and risk
wileyonlinelibrary.com/ETC
© 2022 SETAC
1536
Environmental Toxicology and Chemistry, 2022;41:1520-1539—J.H. Olker et al.
assessments. The ECOTOX Knowledgebase continues to provide needed information for decision support in ecotoxicity, and it is utilized by a wide-range of governmental, academic, and industry researchers and risk assessors; however, there was a recognized need to update the database content and the user interface to keep pace with changes in reported toxicological results. Therefore, extensive updates were recently completed, including a redesigned user interface for web-based access to the Knowledgebase; these new features allow the user to explore and visualize toxicity data, give a variety of data output/ visualization options, and incorporate standardized identifiers (e.g., DTXSIDs) for increased interoperability and reusability.
The ECOTOX Knowledgebase continues to be a flexible resource for empirical toxicity data as the challenge of providing more information at a faster pace to make regulatory decisions is addressed. The primary focus continues to be thoroughly curated data from robust reviews of the literature, with transparency in the processes and accessibility of the curated data. The updates extend the utility of ECOTOX with increased efficiencies to locate relevant toxicity data while building on the existing strengths. These strengths include the comprehensiveness and transparency of the literature search and review processes, the manual curation of detailed information from each relevant study, the inclusion of all measured endpoints capturing both adverse effects and no-effects data, and the extensiveness of toxicity data housed in the Knowledgebase. This ever-growing and evolving database will continue to add applicable and relevant toxicity data while working toward increased efficiency, transparency, and interoperability to support environmental research and risk assessment.
Supporting Information—The Supporting Information is available on the Wiley Online Library at https://doi.org/10.1002/ etc.5324.
Acknowledgment—We gratefully acknowledge the invaluable contributions of multiple USEPA and contract contributors to ECOTOX over the many years that it has been in development. We also recognize and acknowledge the teams at the USEPA's National Health and Environmental Effect Research Laboratory who developed the initial parent databases that were incorporated into ECOTOX (AQUIRE at Mid-Continent Ecology Division, PHYTOTOX and TERRETOX at the Western Ecology Division). We thank our management for ongoing support of this project. In addition, we recognize and thank our USEPA and international colleagues for working with us for improved interoperability. We greatly appreciate the comments and suggestions from G. Ankley (USEPA Office of Research and Development Center for Computational Toxicology) and M. Angrish (USEPA Office of Research and Development Center for Public Health and Environmental Assessment) on an earlier draft of this manuscript. Finally, we thank the USEPA Program Offices and Regions and multiple other end users for their feedback and suggestions to improve the ECOTOX processes and user interface. This work was supported  by the  USEPA Office of Research and
Development and multiple other USEPA offices for targeted literature searches and reviews. Initial development of the unified interface was funded in 1995 through the US Department of Defense's Strategic Environmental Research and Development Program.
Disclaimer—The views expressed in this paper are those of the authors and do not necessarily reflect the views or policies of the US Environmental Protection Agency, nor does the mention of trade names or commercial products indicate endorsement by the federal government.
Author Contributions Statement—Jennifer H. Olker: Conceptualization; Methodology; Formal analysis; Visualization; Writing—original draft, review, and editing. Colleen M. Elonen: Project administration; Conceptualization; Methodology; Funding acquisition; Writing—review and editing. Christine L. Russom: Project administration; Conceptualization; Methodology; Funding acquisition; Writing—review and editing. Anne Pilli: Project administration; Conceptualization; Data curation; Methodology; Writing—review and editing. Arne Anderson: Data curation; Methodology; Writing—review and editing. Brian Kinziger: Data curation; Methodology; Writing—review and editing. Anita Pomplun: Data curation; Methodology; Writing— review and editing. Stephen Erickson: Data curation; Software; Writing—review and editing. Michael Skopinski: Data curation; Software; Writing—review and editing. Carlie A. LaLone: Conceptualization; Writing—review and editing. Dale Hoff: Supervision; Conceptualization; Methodology; Formal analysis; Visualization; Funding acquisition; Writing—original draft, review, and editing.
Data Availability Statement—Data pertaining to the present study are available on the ECOTOX website: www.epa.gov/ ecotox and at https://edg.epa.gov/metadata/catalog/main/ home.page. Data, associated metadata, and calculation tools are also available from the corresponding author (olker. jennifer@epa.gov).
REFERENCES
Alexander, A. C, Luis, A. T., Culp, J. M., Baird, D. J., & Cessna, A. J. (2013). Can nutrients mask community responses to insecticide mixtures? Eco-toxicology, 22(7), 1085-1100. https://doi.org/10.1007/s10646-013-1096-3
Bacey, J., & Spurlock, F. (2007). Biological assessment of urban and agricultural streams in the California Central Valley. Environmental Monitoring and Assessment, 130(1-3), 483^193. https://doi.org/10.1007/ s10661 -006-9438-8
Baker, N., Knudsen, T., & Williams, A. (2017). Abstract Sifter: A comprehensive front-end system to PubMed. FWOOResearch, 6, Article 2164. https://doi.Org/10.12688/f1000research.12865.1
Bermúdez-Saldaňa, J. M., Escuder-Gilabert, L, Medina-Hernández, M. J., Villanueva-Camaňas, R. M., & Sagrado, S. (2005). Modelling bio-concentration of pesticides in fish using biopartitioning micellar chromatography. Journal of Chromatography A, 1063(1-2), 153-160. https:// doi.org/10.1016/j.chroma.2004.11.074
Bettiol, C, De Vettori, S., Minervini, G., Zuccon, E., Marchetto, D., Ghir-ardini, A. V., & Argese, E. (2016). Assessment of phenolic herbicide toxicity and mode of action by different assays. Environmental Science and Pollution Research International, 23(8), 7398-7408. https://doi.org/ 10.1007/s11356-015-5958-5
© 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxicological test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1537
Bezirci, G., Akkas, S. B., Rinke, K., Yildirim, F., Kalaylioglu, Z., Severcan, F., & Beklioglu, M. (2012). Impacts of salinity and fish-exuded kairomone on the survival and macromolecular profile of Daphnia pulex. Eco-toxicology, 21(2), 601-614. https://doi.org/10.1007/s10646-011-0820-0
Bielmyer, G. K., Tomasso, J., & Klaine, S. J. (2006). Physiological responses of hybrid striped bass to aqueous copper in freshwater and saltwater. Archives of Environmental Contamination and Toxicology, 50(4), 531-538. https://doi.Org/10.1007/s00244-005-0131 -7
Brock, T. C, & Van Wijngaarden, R. P. (2012). Acute toxicity tests with Daphnia magna, Americamysis bahia, Chironomus riparius and Gam-marus pulex and implications of new EU requirements for the aquatic effect assessment of insecticides. Environmental Science and Pollution Research International, 19(8), 3610-3618. https://doi.org/10.1007/ s11356-012-0930-0
Claesson, A., & Törnqvist, L. (1988). The toxicity of aluminium to two acido-tolerant green algae. Water Research, 22(8), 977-983.
Connors, K. A., Beasley, A., Barron, M. G., Belanger, S. E., Bonnell, M., Brill, J. L, de Zwart, D., Kienzler, A., Krailler, J., Otter, R., Phillips, J. L, & Embry, M. R.. Creation of a curated aquatic toxicology database: En-viroTox. Environmental Toxicology and Chemistry, (2019) 38(5), 1062-1073. https://doi.Org/10.1002/etc.4382
dAmora, M., & Giordani, S. (2018). The utility of zebrafish as a model for screening developmental neurotoxicity. Frontiers in Neuroscience, 12, Article 976. https://doi.org/10.3389/fnins.2018.00976
Dardenne, F., Nobels, I., De Coen, W., & Blust, R. (2007). Dose-response relationships and statistical performance of a battery of bacterial gene profiling assays. Applied Microbiology and Biotechnology, 75(1), 223-234. https://doi.org/10.1007/s00253-006-0808-5
Demetrio, P. M., Bulus Rossini, G. D., Bonetto, C. A., & Ronco, A. E. (2012). Effects of pesticide formulations and active ingredients on the coelen-terate Hydra attenuata (Pallas, 1766). Bulletin of Environmental Contamination and Toxicology, 88(1), 15-19. https://doi.org/10.1007/ s00128-011-0463-0
Domfeld, C. B., Rodgher, S., Negri, R. G., Espindola, E., & Daam, M. A.
(2019) . Chironomus sancticaroli (Diptera, Chironomidae) as a sensitive tropical test species in laboratory bioassays evaluating metals (copper and cadmium) and field testing. Archives of Environmental Contamination and Toxicology, 76(1), 42-50. https://doi.org/10.1007/ S00244-018-0575-1
Dulio, V., Koschorreck, J., van Bavel, B., van den Brink, P., Hollender, J., Munthe, J., Schlabach, M., Aalizadeh, R., Agerstrand, M., Ahrens, L, Allan, I., Alygizakis, N., Barcelo', D., Bohlin-Nizzetto, P., Boutroup, S., Brack, W., Bressy, A., Christensen, J. H., Cirka, L, ... Slobodnik, J.
(2020) . The NORMAN association and the European Partnership for Chemicals Risk Assessment (PARC): Let's cooperate! Environmental Sciences Europe, 32(1), Article 100. https://doi.org/10.1186/s12302-020-00375-w
Etterson, M. (2020). User's manual: SSD Toolbox (Ver 1.0, EPA/600/R-18/1). https://www.epa.gov/chemical-research/species-sensitivity-distribution-ssd-toolbox
European Chemicals Agency. (2017). The use of alternatives to testing on animals forthe REACH regulation reference (ECHA-17-R-02-EN). https:// doi.org/10.2823/023078
Fai, P. B., & Grant, A. (2010). An assessment of the potential of the microbial assay for risk assessment (MARA) for ecotoxicological testing. Ecotox-icology, 19(8), 1626-1633. https://doi.org/10.1007/s10646-010-0548-2
Fay, K. A., Mingoia, R. T., Goeritz, I., Nabb, D. L, Hoffman, A. D., Ferrell, B. D., Peterson, H. M., Nichols, J. W., Segner, H., & Han, X. (2014). Intra-and interlaboratory reliability of a cryopreserved trout hepatocyte assay forthe prediction of chemical bioaccumulation potential. Environmental Science & Technology, 48(14), 8170-8178. https://doi.org/10.1021/ es500952a
Fay, K. A., Villeneuve, D. L, LaLone, C. A., Song, Y., Tollefsen, K. E., & Ankley, G. T. (2017). Practical approaches to adverse outcome pathway development and weight-of-evidence evaluation as illustrated by ecotoxicological case studies. Environmental Toxicology and Chemistry, 36(6), 1429-1449. https://doi.org/10.1002/etc.3770
Federhen, S. (2012). The NCBI taxonomy database. Nucleic Acids Research, 40(D1), D136-43. https://doi.org/10.1093/nar/gkr1178
George, T. K., & Liber, K. (2007). Laboratory investigation of the toxicity and interaction of pesticide mixtures in Daphnia magna. Archives of Environmental Contamination and Toxicology, 52(1), 64-72. https://doi.org/ 10.1007/s00244-005-0271 -9
Grulke, C. M., Williams, A. J., Thillanadarajah, I., & Richard, A. M. (2019). EPA's DSSTox database: History of development of a curated chemistry resource supporting, computational toxicology research. Computational Toxicology, 12, Article 100096. https://doi.Org/10.1016/j.comtox.2019. 100096
Guzelian, P. S., Victoroff, M. S., Halmes, N. C, James, R. C, & Guzelian, C. P. (2005). Evidence-based toxicology: A comprehensive framework for causation. Human & Experimental Toxicology, 24(A), 161-201. https://doi.org/10.1191/0960327105ht517oa
Guzman, F. T., Gonzalez, F. J., & Martinez, R. R. (2010). Implementing Le-cane quadridentata acute toxicity tests to assess the toxic effects of selected metals (Al, Fe and Zn). Ecotoxicology and Environmental Safety, 73(3), 287-295. https://doi.Org/10.1016/j.ecoenv.2009.10.006
Herrmann, K., Pistollato, F., & Stephens, M. L. (2019). Beyond the 3Rs: Expanding the use of human-relevant replacement methods in biomedical research. ALTEX: Alternatives to Animal Experimentation, 36(3), 343-352. https://doi.org/10.14573/altex.1907031
Hoffmann, S., de Vries, R., Stephens, M. L., Beck, N. B., Dirven, H., Fowle, J. R., 3rd, Goodman, J. E., Hartung, T., Kimber, I., Lalu, M. M., Thayer, K., Whaley, P., Wikoff, D., & Tsaioun, K. (2017). A primer on systematic reviews in toxicology. Archives of Toxicology, 91(7), 2551-2575. https:// doi.org/10.1007/s00204-017-1980-3
Hoffmann, S., & Hartung, T. (2006). Toward an evidence-based toxicology. Human & Experimental Toxicology, 25(9), 497-513. https://doi.org/10. 1191 /0960327106het648oa
Howard, B. E., Phillips, J., Miller, K., Tandon, A., Mav, D., Shah, M. R., Holmgren, S., Pelch, K. E., Walker, V., Rooney, A. A., Macleod, M., Shah, R., & Thayer, K. (2016). SWIFT-Review: A text-mining workbench for systematic review. Systematic Reviews, 5, 87. https://doi.org/10.1186/ s13643-016-0263-z
Howard, B. E., Phillips, J., Tandon, A., Maharana, A., Elmore, R., Mav, D., Sedykh, A., Thayer, K., Merrick, A., Walker, V., Rooney, A., & Shah, R. R. (2020). SWIFT-Active Screener: Accelerated document screening through active learning and integrated recall estimation. Environment International, 138, Article 105623. https://doi.Org/10.1016/j.envint.2020.105623
Interagency Coordinating Committee on the Validation of Alternative Methods. (2018). A strategic roadmap for establishing new approaches to evaluate the safety of chemicals and medical products in the United States, https://ntp.niehs.nih.gov/go/iccvam-rdmp
Ives, C, Campia, I., Wang, R. L, Wittwehr, C, & Edwards, S. (2017). Creating a structured AOP knowledgebase via ointology-based annotations. Applied In Vitro Toxicology, 3(4), 298-311. https://doi.org/10. 1089/aivt.2017.0017
Johnson, B. M., Kanagy, L. E., Rodgers, J. H., & Castle, J. W. (2008). Chemical, physical, and risk characterization of natural gas storage produced waters. Water, Air, and Soil Pollution, 191, 33-54. https://doi. org/10.1007/s11270-007-9605-8
Kaltenhauser, J., Kneuer, C, Marx-Stoelting, P., Niemann, L, Schubert, J., Stein, B., & Solecki, R. (2017). Relevance and reliability of experimental data in human health risk assessment of pesticides. Regulatory Toxicology and Pharmacology, 88, 227-237. https://doi.0rg/IO.IOI6/j.yrtph. 2017.06.010
Klimisch, H. J., Andreae, M., & Tillmann, U. (1997). A systematic approach for evaluating the quality of experimental toxicological and ecotoxicological data. Regulatory Toxicology and Pharmacology, 25(1), 1-5. https://doi.org/10.1006/rtph.1996.1076
Kostich, M. S., Flick, R. W., Batt, A. L, Mash, H. E., Boone, J. S., Furlong, E. T., Kolpin, D. W., & Glassmeyer, S. T. (2017). Aquatic concentrations of chemical analytes compared to ecotoxicity estimates. Science of the Total Environment, 579, 1649-1657. https://doi.0rg/IO.IOI6/j.scitotenv. 2016.06.234
Krewski, D., Andersen, M. E., Tyshenko, M. G., Krishnan, K., Hartung, T., Boekelheide, K., Wambaugh, J. F., Jones, D., Whelan, M., Thomas, R., Yauk, C, Barton-Maclaren, T., & Cote, I. (2020). Toxicity testing in the 21st century: Progress in the past decade and future perspectives. Archives of Toxicology, 94(1), 1-58. https://doi.org/10.1007/s00204-019-02613-4
Krewski, D., Westphal, M., Andersen, M. E., Paoli, G. M., Chiu, W. A., Al-Zoughool, M., Croteau, M. C, Burgoon, L. D., & Cote, I. (2014). A framework for the next generation of risk science. Environmental Health Perspectives, 122(8), 796-805. https://doi.org/10.1289/ehp. 1307260
LaLone, C. A., Villeneuve, D. L, Lyons, D., Helgen, H. W., Robinson, S. L., Swintek, J. A., Saari, T. W., & Ankley, G. T. (2016). Editor's highlight:
wileyonlinelibrary.com/ETC
© 2022 SETAC
1538
Environmental Toxicology and Chemistry, 2022;41:1520-1539—J.H. Olker et al.
Sequence Alignment to Predict Across Species Susceptibility (SeqA-PASS): A web-based tool for addressing the challenges of cross-species extrapolation of chemical toxicity. Toxicological Sciences, 153(2), 228-245. https://doi.org/10.1093/toxsci/kfw119
LeBlanc, H. M., Culp, J. M., Baird, D. J., Alexander, A. C, & Cessna, A. J. (2012). Single versus combined lethal effects of three agricultural insecticides on larvae of the freshwater insect C/iironomus dilutus. Archives of Environmental Contamination and Toxicology, 63(3), 378-390. https://doi.Org/10.1007/s00244-012-9777-0
Leboulanger, C, Bouvy, M., Carre, C, Cecchi, P., Amalric, L, Bouchez, A., Pagano, M., & Sarazin, G. (2011). Comparison of the effects of two herbicides and an insecticide on tropical freshwater plankton in microcosms. Arc/lives of Environmental Contamination and Toxicology, 61(4), 599-613. https://doi.org/10.1007/s00244-011-9653-3
Lee, K. E., Langer, S. K., Menheer, M. A., Hansen, D. S., Foreman, W. T., Furlong, E. T., Jorgenson, Z. G., Choy, S. J., Moore, J. N., Banda, J., & Gefell, D. J. (2015). Chemicals of emerging concern in water and bottom sediment in the Great Lakes Basin, 2012—Collection methods, analytical methods, quality assurance, and study data (Data Series 910). US Geological Survey, https://doi.org/10.3133/ds910
Lee, Y. G., Hwang, S. H., & Kim, S. D. (2006). Predicting the toxicity of substituted phenols to aquatic species and its changes in the stream and effluent waters. Archives of Environmental Contamination and Toxicology, 50(2), 213-219. https://doi.org/10.1007/s00244-004-1259-6
Liu, A. X., Lang, Y. H., Xue, L. D., Liao, S. L, & Zhou, H. (2009). Probabilistic ecological risk assessment and source apportionment of polycyclic aromatic hydrocarbons in surface sediments from Yellow Sea. Bulletin of Environmental Contamination and Toxicology, 83(5), 681-687. https:// doi.org/10.1007/s00128-009-9870-x
Moermond, C, Beasley, A., Breton, R., Junghans, M., Laskowski, R., Solomon, K., & Zahner, H. (2017). Assessing the reliability of ecotoxicological studies: An overview of current needs and approaches. Integrated Environmental Assessment and Management, 13(4), 640-651. https://doi.org/10.1002/ieam.1870
Moermond, C. T., Kase, R., Korkaric, M., & Agerstrand, M. (2016). CRED: Criteria for reporting and evaluating ecotoxicity data. Environmental Toxicology & Chemistry, 35(5), 1297-1309. https://doi.org/10.1002/ etc.3259
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G.; PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), Article e1000097. https://doi. org/10.1371/journal.pmed.1000097
Nam, S. H., Lee, W. M., & An, Y. J. (2012). Derivation of ecological protective concentration using the probabilistic ecological risk assessment applicable for Korean water environment: (I) Cadmium. Toxicological Research, 28(2), 129-137. https://doi.Org/10.5487/TR.2012.28.2.129
National Research Council. (2007). Toxicity testing in the 21st century: A vision and a strategy. National Academies Press.
National Toxicology Program. (2019). Handbook for conducting a literature-based health assessment using OHAT approach for systematic review and evidence integration. Retrieved January 13, 2022, from: https://ntp. niehs.nih.gov/ntp/ohat/pubs/handbookmarch2019_508.pdf
Organisation for Economic Co-operation and Development. (2017). Guidance document for describing non-guideline in vitro test methods. Series on Testing and Assessment, No. 211 (ENV/JM/MONO(2014)35). https://doi.org/10.1787/9789264274730-en
Organisation for Economic Co-operation and Development. (2018). Guidance document on Good In Vitro Method Practices (GIVIMP). Series on Testing and Assessment, No. 286 (ENV/JM/MONO(2018)19). https://doi. org/10.1787/9789264304796-en
Ownby, D. R., & Newman, M. C. (2003). Advances in quantitative ion character-activity relationships (QICARs): Using metal-ligand binding characteristics to predict metal toxicity. Quantitative Structure-Activity Relationships, 22, 241-246.
Parish, S. T., Aschner, M., Casey, W., Corvaro, M., Embry, M. R., Fitzpatrick, S., Kidd, D., Kleinstreuer, N. C, Lima, B. S., Settivari, R. S., Wolf, D. C, Yamazaki, D., & Boobis, A. (2020). An evaluation framework for new approach methodologies (NAMs) for human health safety assessment. Regulatory Toxicology and Pharmacology, 112, Article 104592. https:// doi.org/10.1016/j.yrtph.2020.104592
Ra, J. S., Lee, B. C, Chang, N. I., & Kim, S. D. (2008). Comparative whole effluent toxicity assessment of wastewater treatment plant effluents using Daphnia magna. Bulletin of Environmental Contamination and Toxicology, 80(3), 196-200. https://doi.org/10.1007/s00128-007-9344-y
Ralston-Hooper, K., Hardy, J., Hahn, L., Ochoa-Acuria, H., Lee, L. S., Mollenhauer, R., & Sepülveda, M. S. (2009). Acute and chronic toxicity of atrazine and its metabolites deethylatrazine and deisopropylatrazine on aquatic organisms. Ecotoxicology, 18(7), 899-905. https://doi.org/10. 1007/s10646-009-0351-0
Rial, D., Beiras, R., Vazquez, J. A., & Murado, M. A. (2010). Acute toxicity of a shoreline cleaner, CytoSol, mixed with oil and ecological risk assessment of its use on the Galician coast. Archives of Environmental Contamination and Toxicology, 59(3), 407-416. https://doi.org/10.1007/ S00244-010-9492-7
Rosenbaum, R. K., Bachmann, T. M., Gold, L. S., Huijbregts, M. A. J., Jolliet, O., Juraske, R., Koehler, A., Larsen, H. F., MacLeod, M., Margni, M., McKone, T. E., Payet, J., Schuhmacher, M., van de Meent, D., & Hauschild, M. Z. (2008). USEtox—The UNEP-SETAC toxicity model: Recommended characterisation factors for human toxicity and freshwater ecotoxicity in life cycle impact assessment. International Journal of Life Cycle Assessment, 13(7), 532-546. https://doi.org/10.1007/s11367-008-0038-4
Roth, N., Zilliacus, J., & Beronius, A. (2021). Development of the SciRAP approach for evaluating the reliability and relevance of in vitro toxicity data. Frontiers in Toxicology, 3, Article 746430. https://doi.org/10.3389/ ftox.2021.746430
Rüden, C, Adams, J., Agerstrand, M., Brock, T. C, Poulsen, V., Schlekat, C. E., Wheeler, J. R., & Henry, T. R. (2017). Assessing the relevance of ecotoxicological studies for regulatory decision making. Integrated Environmental Assessment and Management, 13(4), 652-663. https://doi. org/10.1002/ieam.1846
Russom, C. L., LaLone, C. A., Villeneuve, D. L., & Ankley, G. T. (2014). Development of an adverse outcome pathway for acetylcholinesterase inhibition leading to acute mortality. Environmental Toxicology and Chemistry, 33(10), 2157-2169. https://doi.org/10.1002/ etc.2662
San-Segundo, L, Guimaräes, L.Torija, C. F., Beiträn, E. M., Guilhermino, L., & Pablos, M. V. (2016). Alterations in gene expression levels provide early indicators of chemical stress during Xenopus laevis embryo development: A case study with perfluorooctane sulfonate (PFOS). Ecotoxicology and Environmental Safety, 127, 51-60. https://doi.org/10. 1016/j.ecoenv.2016.01.005
Schneider, K., Schwarz, M., Burkholder, I., Kopp-Schneider, A., Edler, L., Kinsner-Ovaskainen, A., Härtung, T., & Hoffmann, S. (2009). "ToxR-Tool," a new tool to assess the reliability of toxicological data. Toxicology Letters, 189(2), 138-144. https://doi.Org/10.1016/j.toxlet.2009. 05.013
Schuler, L. J., Hoang, T. C, & Rand, G. M. (2008). Aquatic risk assessment of copper in freshwater and saltwater ecosystems of south Florida. Ecotoxicology, 17(7), 642-659. https://doi.org/10.1007/s10646-008-0236-7
Shuhaimi-Othman, M., Nadzifah, Y., Umirah, N. S., & Ahmad, A. K. (2012). Toxicity of metals to an aquatic worm, nais elinguis (Oligochaeta, Nai-didae). Research Journal of Environmental Toxicology, 6(4), 122-132. https://doi.org/10.3923/rjet.2012.122.132
Society of Environmental Toxicology and Chemistry. (2019). Recommended minimum reporting information for environmental toxicity studies [Technical issue paper].
Sorensen, M. T., Conder, J. M., Fuchsman, P. C, Martello, L. B., & Wenning, R. J. (2007). Using a sediment quality triad approach to evaluate benthic toxicity in the lower Hackensack River, New Jersey. Archives of Environmental Contamination and Toxicology, 53(1), 36-49. https://doi.org/ 10.1007/s00244-006-0165-5
Stadlinger, N., Berg, H., Van den Brink, P. J., Tarn, N. T., & Gunnarsson, J. S. (2018). Comparison of predicted aquatic risks of pesticides used under different rice-farming strategies in the Mekong delta, Vietnam. Environmental Science and Pollution Research International, 25(14), 13322-13334. https://doi.Org/10.1007/s11356-016-7991 -4
Stephens, M. L., Akgün-Ölmez, S. G., Hoffmann, S., de Vries, R., Flick, B., Härtung, T., Lalu, M., Maertens, A., Witters, H., Wright, R., & Tsaioun, K. (2018). Adaptation of the systematic review framework to the assessment of toxicological test methods: Challenges and lessons learned with the zebrafish embryotoxicity test. Toxicological Sciences, 171(1), 56-68. https://doi.Org/10.1093/toxsci/kfz128
Stephens, M. L, Andersen, M., Becker, R. A., Betts, K., Boekelheide, K., Carney, E., Chapin, R., Devlin, D., Fitzpatrick, S., Fowle, J. R., 3rd, Harlow, P., Härtung, T., Hoffmann, S., Holsapple, M., Jacobs, A., Judson, R., Naidenko, O., Pastoor, T., Patlewicz, G., ... Zurlo, J. (2013). Evidence-based toxicology for the 21st century: Opportunities and
© 2022 SETAC
wileyonlinelibrary.com/ETC
ECOTOX Knowledgebase: A database of ecotoxlcologlcal test results—Environmental Toxicology and Chemistry, 2022;41:1520-1539
1539
challenges. ALTEX: Alternativea to Animal Experimentation, 30(1), 74-103. https://doi.Org/10.14573/altex.2013.1.074
Stephens, M. L, Betts, K., Beck, N. B., Cogliano, V., Dickersin, K., Fitzpatrick, S., Freeman, J., Gray, G., Härtung, T., McPartland, J., Rooney, A. A., Scherer, R. W., Verloo, D., & Hoffmann, S. (2016). The emergence of systematic review in toxicology. Toxicological Science, 152(1), 10-16. https://doi.org/10.1093/toxsci/kfw059
Texas Commission on Environmental Quality. (2017). TCEQ guidelines for systematic review and evidence integration [White paper]. Retrieved January 13, 2022, from: https://www.tceq.texas.gov/assets/public/ implementation/tox/dsd/whitepaper/srguidelines.pdf
Thayer, K. A., Wolfe, M. S., Rooney, A. A., Boyles, A. L, Bucher, J. R., & Birnbaum, L. S. (2014). Intersection of systematic review methodology with the NIH reproducibility initiative. Environmental Health Perspectives, 122(7), A176-A177. https://doi.org/10.1289/ehp.1408671
Thomas, R. S., Bahadori, T., Buckley, T. J., Cowden, J., Deisenroth, C, Dionisio, K. L, Frithsen, J. B., Grulke, C. M., Gwinn, M. R., Harrill, J. A., Higuchi, M., Houck, K. A., Hughes, M. F., Hunter, E. S., Isaacs, K. K., Judson, R. S., Knudsen, T. B., Lambert, J. C, Linnenbrink, M., ... Williams, A. J. (2019). The next generation blueprint of computational toxicology at the U.S. Environmental Protection Agency. Toxicological Sciences, 169(2), 317-332. https://doi.org/10.1093/toxsci/kfz058
Tousova, Z., Froment, J., Oswald, P., Slobodník, J., Hilscherova, K., Thomas, K. V., Tollefsen, K. E., Reid, M., Langford, K., & Blaha, L. (2018). Identification of algal growth inhibitors in treated waste water using effect-directed analysis based on non-target screening techniques. Journal of Hazardous Materials, 358, 494-502. https://doi.Org/10.1016/j.jhazmat. 2018.05.031
Traoré, H., Crouzet, O., Mamy, L, Sireyjol, C, Rossard, V., Servien, R., Latrille, E., Martin-Laurent, F., Patureau, D., & Benoit, P. (2018). Clustering pesticides according to their molecular properties, fate, and effects by considering additional ecotoxicological parameters in the TyPol method. Environmental Science and Pollution Research International, 25(5), 4728-4738. https://doi.org/10.1007/s11356-017-0758-8
US Environmental Protection Agency. (2014). Next generation risk assessment: Incorporation of recent advances in molecular, computational, and systems biology (Final report EPA/600/R-14/004). Retrieved January 13, 2022, from: https://ofmpub.epa.gov/eims/eimscomm.getfile? p_download_id=520491
US Environmental Protection Agency. (2020a). New approach methods work plan: Reducing use of animals in chemical testing (EPA 615B2001). Retrieved January 13, 2022, from: https://www.epa.gov/sites/default/ files/2020-06/documents/epa_nam_work_plan.pdf
US Environmental Protection Agency. (2020b). ORD staff handbook for developing IRIS assessments (Public comment draft, EPA/600/R-20/137). Retrieved January 13, 2022, from: https://ofmpub.epa.gov/eims/ eimscomm.getfile?p_download_id=541571
Villeneuve, D. L, Coady, K., Escher, B. L, Mihaich, E., Murphy, C. A., Schlekat, T., & Garcia-Reyero, N. (2019). High-throughput screening and environmental risk assessment: State of the science and emerging applications. Environmental Toxicology and Chemistry, 38(1), 12-26.
Voigt, K., Brüggemann, R., & Pudenz, S. (2004). Chemical databases evaluated by order theoretical tools. Analytical & Bioanalytical Chemistry, 380(3), 467-474. https://doi.org/10.1007/s00216-004-2794-8
Wang, R. L., Edwards, S., & Ives, C. (2019). Ontology-based semantic mapping of chemical toxicities. Toxicology, 412, 89-100. https://doi.org/ 10.1016/j.tox.2018.11.005
Watford, S., Edwards, S., Angrish, M., Judson, R. S., & Paul Friedman, K. (2019). Progress in data interoperability to support computational toxicology and chemical safety evaluation. Toxicology and Applied Pharmacology, 380, Article 114707. https://doi.Org/10.1016/j.taap.2019.114707
Watford, S., Pham, L. L, Wignall, J., Shin, R., Martin, M. T., & Friedman, K. P. (2019). ToxRefDB version 2.0: Improved utility for predictive and retrospective toxicology analyses. Reproductive Toxicology, 89, 145-158. https://doi.Org/10.1016/j.reprotox.2019.07.012
Whaley, P., Edwards, S. W., Kraft, A., Nyhan, K., Shapiro, A., Watford, S., Wattam, S., Wolffe, T., & Angrish, M. (2020). Knowledge organization systems for systematic chemical assessments. Environmental Health Perspectives, 128(12), Article 125001. https://doi.org/10.1289/ EHP6994
Wikoff, D. S., & Miller, G. W. (2018). Systematic reviews in toxicology. Toxicological Sciences, 163(2), 335-337. https://doi.org/10.1093/toxsci/ kfy109
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J. W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., ... Möns, B. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3, Article 160018. https://doi.org/10.1038/sdata.2016.18
Williams, A. J., Grulke, C. M., Edwards, J., McEachran, A. D., Mansouri, K, Baker, N. C, Patlewicz, G., Shah, I., Wambaugh, J. F., Judson, R. S., & Richard, A. M. (2017). The CompTox Chemistry Dashboard: A community data resource for environmental chemistry. Journal of Cheminformatics, 9(1), Article 61. https://doi.org/10.1186/s13321-017-0247-6
Williams, E. S., Berninger, J. P., & Brooks, B. W. (2011). Application of chemical toxicity distributions to ecotoxicology data requirements under REACH. Environmental Toxicology and Chemistry, 30(8), 1943-1954. https://doi.org/10.1002/etc.583
Wolffe, T., Whaley, P., Halsall, C, Rooney, A. A., & Walker, V. R. (2019). Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environment International, 130, 104871. https://doi.Org/10.1016/j.envint.2019.05.065
Woodruff, T. J., & Sutton, P. (2014). The Navigation Guide systematic review methodology: A rigorous and transparent method for translating environmental health science into better health outcomes. Environmental Health Perspectives, 122(10), 1007-1014. https://doi.org/10.1289/ehp. 1307175
Yang, Y., Yun, X., Liu, M., Jiang, Y., Li, Q. X., & Wang, J. (2014). Concentrations, distributions, sources, and risk assessment of organochlorine pesticides in surface water of the East Lake, China. Environmental Science and Pollution Research International, 21(A), 3041-3050. https://doi. org/10.1007/s! 1356-013-2269-6
wileyonlinelibrary.com/ETC
© 2022 SETAC