SIGNAL MANAGEMENT · implementation considerations within a generic safety signal management...

SIGNAL MANAGEMENT: CURRENT LANDSCAPE AND

CONSIDERATIONS FOR BEST PRACTICES

A U T H O R S : A n t o n i W i s n i e w s k i ( A s t r a Z e n e c a ) , A n d r e s G o m e z ( B r i s t o l - M y e r s S q u i b b C o m p a n y ) , J e r e m y J o k i n e n ( A b b V i e ) , K a r o l L a C r o i x ( G l a x o S m i t h K l i n e ) , A n j u G a r g ( S a n o f i ) , N e a l G r a b o w s k i ( A m g e n ) , R i c h a r d H e r m a n n ( A s t r a Z e n e c a )

P U B L I S H E D : A P R I L 2 0 2 0

Advancing Safety Analytics Initiative

Signal Management: Current Landscape and Considerations for Best Practices

Copyright © 2020 TransCelerate BioPharma Inc. All Rights Reserved. 2

TABLE OF CONTENTS Table of Tables Table of Figures List of Abbreviations Abstract Introduction Signal Management Process Framework Overview and Definitions

1. Timelines 2. Prioritization 3. Division of Responsibilities 4. Hierarchy of Evidence

Detection Stage

1. Safety Observations 2. Data Sources

a. EudraVigilance Data Analysis System (EVDAS) b. FDA Adverse Event Reporting System (FAERS) and VigiBase® c. Sponsor Database d. Other Data Sources

3. Detection a. Aggregate Statistical Methods b. Considerations for Refining Methods and Adjusting

Thresholds c. External Databases

4. Triage




TABLE OF CONTENTS CONT. Evaluation Stage

1. Validation 2. Signal Assessment

Action Stage

1. Actionable Safety Outcomes 2. Documentation and Tracking

Discussion Conclusions References Acknowledgments




TABLE OF TABLES

Table 1. Databases for Safety Data Table 2. Summary of Industry Survey Results: Signals and Primary Source of Information Triggering a Signal Table 3. Evaluation Stage: Distinct Triage and Validation versus Integrated Validation Table 4. Actionable Safety Outcomes Table 5. Recommended Minimum Items for Documentation Throughout the SM Process Table 6. Consideration for Establishing or Re-appraising a Signal Management Process

TABLE OF FIGURES

Figure 1. Signal Management (SM) Decision Framework




LIST OF ABBREVIATIONS

ABBREVIATION DEFINITION

ADR adverse drug reaction

AE adverse event

DEC drug-event combination

EEA European Economic Area

EMA European Medicines Agency

eRMR electronic reaction monitoring report

EU European Union

EV EudraVigilance

EVDAS EudraVigilance Data Analysis System

FAERS FDA Adverse Event Reporting System

FDA Food and Drug Administration

FOIA Freedom of Information Act

GCP Good Clinical Practices

GVP Good Pharmacovigilance Practices

HCP healthcare professional

ICH International Council for Harmonisation

ICSR Individual Case Safety Reports

IMI PROTECT Pharmacoepidemiological Research on Outcomes of Therapeutics by a European Consortium

MAH marketing authorization holder

MedDRA Medical Dictionary for Regulatory Activities

NPV negative predictive value

PV pharmacovigilance

PPV positive predictive value

PSMF PV system master file

PSUR Periodic Safety Update Report

PT preferred term

ROR reporting odds ratio

SDR signals of disproportionate reporting

SM signal management

SMQ Standard MedDRA Queries

WHO World Health Organization




ABSTRACT

Signal management continues to evolve as the practices and expectations of regulatory authorities change, and advances in data science and digital technology are adopted by the pharmacovigilance community. Although the regulatory environment encourages some standardization of signal management, individual company practices vary, and divergent interpretations of terminology and procedural requirements often hinder identification and discussion of best practices for detecting, strengthening, and assessing safety signals. This paper reviews the current landscape of signal management and proposes a simplified, end-to-end framework while highlighting some best practices. We have identified a simplified, general framework for describing end-to-end signal management that captures commonalities and variations across the industry, which allows for variances in definitions and procedural implementation at each organization. Finally, we advocate further research into “risk-based” or “value-based” approaches to improve signal management through identification of high- and low-value data/analyses, assignment of value weights to data, appropriate allocation of resources, periodic review of system operating characteristics, and periodic assessment of new data and analytics sources.

INTRODUCTION The world of safety analytics continues to evolve as new sources of information become available and safety databases grow increasingly extensive, both in terms of products and sources of adverse event (AE) reports. Although product marketing authorization holders (MAHs) have specific requirements for reporting AEs and reviewing safety data, they also have an array of similar and diverse options for meeting these requirements. This paper describes a range of implementation considerations within a generic safety signal management framework and highlights recent analytic advances, identifies opportunities for additional advances, and outlines considerations for an appropriate value-based framework to direct resources toward the highest value activities consistent with a product’s lifecycle and understanding of benefit-risk profile. We suggest best practices for some elements and propose future work to establish best practices in other areas. TransCelerate Biopharma, Inc.’s Advancing Safety Analytics team collaborated to understand end-to-end signal management and the unique value or contribution of publicly available data sources. The main goal of both efforts was to inform decision making in the management of safety signals by studying a specific application in signal detection. A further goal was to identify new data sources and novel methodologies that could drive the advancement of safety analytics, further the understanding of product safety profiles, and inform product development. In doing so, we uncovered key learnings regarding the impact of new analytical sources on the signal management process presented below.




SIGNAL MANAGEMENT PROCESS FRAMEWORK OVERVIEW AND DEFINITONS In 2012, Good Pharmacovigilance Practices (GVP) Module IX introduced a distinct set of SM stages.1 The sequence and nomenclature of these stages are based on the setup and shared responsibilities in the European Union (EU) regulatory network. Based on the implementation of GVP Module IX, we aim to describe a generic SM process framework (Figure 1), building on general concepts in knowledge discovery that can be recognized in any pharmacovigilance (PV) setting.1 The actual application of these stages must account for variations in each company’s signal process, driven by the organizational setup and functional responsibilities. Nevertheless, signal management tends to follow a consistent trajectory from safety observation, to signal/validated signal, and finally, to actionable safety outcome. Data sources for safety observations typically include the company safety database, external databases, and biomedical literature. Triage, a preliminary assessment and prioritization of safety observations, is generally confined to the source from which the safety observation was detected but may extend to additional sources (e.g., literature). GVP Module IX provides separate definitions for validation and assessment. In essence, the validation step is considered a preliminary evaluation stage of a numeric or qualitative signal. This step aims to determine if a thorough assessment is required (i.e., whether there is sufficient evidence to warrant a detailed review of all available data sources to permit a conclusion about a causal association between an intervention and an undesirable outcome). Some companies make a distinction between validation and assessment (i.e., implement a 2-step process), whereas other companies have implemented a single-step process, typically referred to as either assessment or evaluation. Similarly, some companies may combine the triage and validation into a single process step. Notwithstanding the number of process steps and the variable interpretations of terminology, all companies need a process that allows for the detection and assessment of safety signals with documentation of decisions at each step. A multi-step process serves an important purpose by introducing “noise-filtering activities,” such as triage and validation, that consume relatively fewer resources per safety observation. These activities can free up time for those safety issues meriting a full assessment, resulting in both a higher quality of that assessment and a net reduction in resources required to manage the overall SM system.

1 The Framework and points addressed herein are provided for informational purposes only. TransCelerate and its

members do not accept any responsibility for any loss of any kind, including loss of revenue, business, anticipated savings

or profits, loss of goodwill or data, or for any indirect consequential loss whatsoever, suffered by any person using this

Framework or acting or refraining from action as a result of the information contained in these materials. Moreover, any

party using the Framework or relying on the information set out in this Paper bears sole and complete responsibility for

not only complying but determining how best to comply with all applicable laws and regulations in all relevant

jurisdictions regarding pharmacovigilance monitoring, reporting, and compliance.




Aspects considered by companies when determining what steps to implement for their respective SM process include:

Timelines Companies ensure that preliminary assessment (i.e., triage and validation) and prioritization are expeditiously conducted and documented after detection while allowing adequate time for detailed review and assessment. Likewise, companies determine the frequency of routine signal detection and assessment. In some cases, the frequency may vary depending on the data source, the life-cycle stage of the product, or the volume of safety observations being generated.

Prioritization Although prioritization will occur throughout the SM process, companies may formally document prioritization at given stages (e.g., to align with GVP Module IX prioritization). Companies with distinct validation and assessment steps may perform prioritization activities after the triage step. The validation step may incorporate a formal (re-) prioritization decision.

Division of Responsibilities Responsibility for executing SM activities and accountability for decisions varies across companies. Generally, however, PV departments tend to be solely responsible for triage. In contrast, teams of an increasingly diverse, cross-functional nature are the norm as the signal proceeds through to full assessment.

Hierarchy of Evidence For companies with distinct validation and assessment steps, the delineation between preliminary and thorough assessment is not easy to define and varies amongst companies. A company may approach validation by considering the signal source (e.g., from health authorities, which are immediately considered as validated and thoroughly assessed), individual case safety reports (ICSR) identification with positive dechallenge or rechallenge requiring a thorough review of data sources before making decisions and the criteria for that review (e.g., at minimum, PV database, literature, and prescribing information for products in similar class). A survey of member companies was conducted to understand the current safety analytics landscape, including opportunities to evaluate common practices. It became clear, however, that companies varied in their SM processes, the main sources of variance being definitions of key process steps and the evidence thresholds used for moving from one step to the next. The insight gained from that survey allows us to now identify a flexible SM process framework to explain these variations and describe its components. Moreover, a model SM process framework might help us understand how companies make decisions to balance strict regulatory requirements with those for which there is some degree of latitude/discretion. By using our SM process framework (Figure 1) and identifying considerations within each of its components (e.g., best practices, areas of latitude, the effectiveness of various detection methods), we believe a value-based approach to safety analytics could emerge that enhances the quality of the information provided by MAHs to health authorities, the efficiency with which MAHs provide this information, and the effectiveness with which companies communicate SM decisions to their stakeholders.




Figure 1. Signal Management (SM) Decision Framework

DETECTION STAGE In our framework, signal detection constitutes the processes for identifying new safety observations from available data sources. It ends with a determination of whether or not the observation is a "signal" (i.e., that the signal requires validation).

Safety Observations Routine signal detection produces safety observations as a result of qualitative or quantitative methods applied to data. Quantitative approaches, mainly predicated on disproportionality analysis or trend analysis, are available in the post-marketing setting when data volumes render a comprehensive review of ICSRs impracticable. A safety observation is information from any source that differs from what is known about a product, either quantitatively or qualitatively. It has the potential to affect the risk profile of that product. A safety observation is preliminary and is not equivalent to a safety signal. A safety observation may or may not become a signal after applying relevant clinical/scientific context and medical judgment.

Data Sources For marketed products, several AE databases are available to be queried and used for the generation of safety observations, as well as providing additional data to support ad hoc analyses into existing safety questions. Table 1 describes commonly used internal and external safety databases.




Table 1. Databases for Safety Data

DATABASE BRIEF DESCRIPTION FEATURES

EudraVigilance Data Analysis System (EVDAS)

The EMA’s database for reporting and analysis of AEs occurring with products under development or approved within the EEA

• AE reporting gateway

• Data since 2001

• EV Data Warehouse & Analysis System

• Nightly updates

• Data-mining results with a pre-specified methodology and format

• Access varies by user affiliation

• Case narratives available to support signal evaluations & other EMA procedures (e.g., periodic reports)

FDA Adverse Event Reporting System (FAERS)

Includes reports from healthcare professionals and patients submitted directly to the FDA in addition to those provided by manufacturers

• Data since 1968

• Incremental datasets released quarterly

• Allows any method and display in any desired format

• Public dashboard launched in 2018, allows limited quantitative analysis by any user

• Narratives only available by FOIA requests

• Substantial time lag for getting FAERS data (e.g., 6 to 9 months)

WHO VigiBase®

A global database of case safety reports from over 130-member countries

• Used broadly in methods research and validation

• Screened routinely by the Uppsala Monitoring Centre

• Quarterly releases

• Choice of methodology and display format

• Data is heavily de-identified, including the use of age ranges, limiting the granularity of the data.

• Narratives are not available.

Sponsor Database

Database of AEs and other safety findings reported for a drug in development or on the market, maintained by the pharmaceutical manufacturer of that drug

• Contains the most comprehensive, complete, and contemporary data for a drug

• Allows each sponsor to choose the methods and frequency best suited for answering specific questions

**AE = adverse event; EEA = European Economic Area; EMA = European Medicines Agency; EV = EudraVigilance; EVDAS = EudraVigilance Data Analysis System; FAERS = FDA Adverse Event Reporting System; FDA = Food and Drug Administration; FOIA = Freedom of Information Act; WHO = World Health Organization

Regarding coding, structuring, and retrieving AEs from these and other safety databases, the Medical Dictionary for Regulatory Activities (MedDRA) is the International Council for Harmonisation (ICH) standard taxonomy. It allows for the extraction of cases at either a very granular (i.e., Preferred Term [PT] level) or broader concept level, including Standard MedDRA Queries (SMQs) that represent medical concepts or areas of safety interest that span the MedDRA taxonomy and arise from signs, symptoms, diagnoses, laboratory, and other test data. There is a trade-off between granularity and the risk of dilution of a safety signal across multiple PTs. Earlier work from Pharmacoepidemiological Research on Outcomes of Therapeutics by a European Consortium (IMI PROTECT) suggests that there may be a loss of timeliness in detecting signals using established statistical methods at anything higher than the PT level.2, 3




EudraVigilance Data Analysis System (EVDAS) EVDAS data can be used as a source in the detection stage. For products in scope of the EMA pilot, EVDAS data must be used as a source in the detection stage. EVDAS data can also be used as support in the evaluation stage for signals identified from other sources. When compared to FAERS and Vigibase, the EVDAS database provides more limited access to MAHs through a static, pre-defined report – the electronic reaction monitoring report (eRMR) that consists essentially of the reporting odds ratio (ROR) as the disproportionality statistic. The eRMR is static in that no additional statistical measures or restrictions can be applied, and the data presented are pre-defined and not customizable. The report provides some limited subgroup statistics by age and geography. Data can only be presented by product at the active ingredient level providing no differentiation between reports from different manufacturers. Events can be restricted to any level of the MedDRA hierarchy or SMQs. There is a legal mandate for European MAHs to monitor eRMRs and download external database cases not already in a sponsor’s database.4 The EMA and national competent authorities routinely screen EVDAS. Sponsor monitoring of the dataset provides insights into class effects and other health authority requests.

FDA Adverse Event Reporting System (FAERS) and VigiBase® FAERS and VigiBase® data can also be used as source material in the detection stage or as support in the evaluation stage for signals identified from other sources. FAERS and VigiBase® datasets are released quarterly for upload to the sponsor system of choice for analysis. This flexibility allows for the application of any disproportionality method and threshold deemed appropriate by the sponsor. Subsets may be created for targeted queries or to avoid confounding factors (e.g., masking by legal cases). The ability to load FAERS and Vigibase® data into an analytic system of choice allows for greater flexibility in the methods and MedDRA hierarchy level for routine signal detection. This option, however, requires evaluating and selecting the most appropriate statistical thresholds and aggregation for the product under surveillance. There is no legal mandate to monitor FAERS or VigiBase® or to reconcile cases to the sponsor database. The FDA routinely monitors FAERS and posts potential signals of serious risk/new safety information on their website in addition to issuing queries to sponsors.

Sponsor Database The sponsor safety database of ICSRs usually provides the most comprehensive and contemporary data for analysis with the application of any selected methodology. The relatively smaller size, compared to external databases, may limit the use of disproportionality methods as reliable effect estimation requires robust background data. When considering what database(s) to monitor on a routine basis, sponsors should consider several factors. The time on market and jurisdiction will determine when or if cases appear in a given database as dictated by the frequency of data releases and the reporting rules of the various databases. Numerous confounding factors exist that are difficult to control for and may impact the utility of statistical methods.




Other Data Sources Published biomedical literature is another mandated source of surveillance for both individual cases and aggregate safety information.1, 5, 6, 7 The overall contribution of literature surveillance to benefit-risk assessment is increasing as literature mining tools improve.8, 9, 10 Furthermore, the biomedical literature can be the major source of new safety information for very mature products for which AE reporting is sparse, particularly as a source of safety information relating to class or drug target effects. This use-case is tempered, however, by the latency of writing and publishing articles. At the same time, the information may become available contemporaneously, or indeed earlier, through other reporting channels. The principal value of literature comes in signal assessments when additional data is required to interrogate the plausibility and mechanism of a suspected adverse drug reaction (ADR). The company's clinical database represents the first source of human safety data during the development of a product. It serves as the sponsor’s baseline when evaluating and contextualizing post-marketing safety data as it is well controlled, structured, and comprehensive. Other company data may help provide context or controls to aggregate signal detection activities. Some examples include non-clinical studies, batch size, and distribution information. Product complaints, which may include device-related issues, can provide additional information for the evaluation of AEs that are indicative of physical delivery system signals (e.g., wrong technique in product usage) versus the medicinal component of the product. Some companies use external observational longitudinal data sources (e.g., health insurance claims databases or electronic health records) for the detection stage; however, given the variable results of studies exploring the value of these data sources, their use here is not universal. Similarly, databases of postings from the most widely used social media platforms have met with mixed results when used for signal detection, such that it has not been possible to recommend these sources for a broad-based approach to the detection stage.11, 12

Detection As described, databases with continually accumulating AE information are maintained by pharmaceutical companies and regulatory authorities globally. The volume of information contained within these databases can make meaningful use of these data challenging, particularly when a drug goes on the market. As a result, methods for mining the aggregated data have been a topic of regulatory, academic, and industry research for over 20 years. Data-mining programs are run regularly or on an ad hoc basis to generate statistical scores for each drug-event combination (DEC) from the complete case data or any subgroup. Disproportionality analysis and trend analysis predicate these quantitative approaches. For further discussion on the use of these quantitative approaches, refer to GVP Annex I.13 At this point in our framework, the product of the various signal generation activities are still safety observations, typically presented as DECs or drug-event pairs. Each DEC is then triaged based on those scores and other relevant information to decide whether to advance it for further evaluation.

Aggregate Statistical Methods The most common approach to mining PV databases is disproportionality analysis. Disproportionality analyses identify drug-event pairs that are reported more frequently for a targeted drug of interest than would be expected. What is “expected” in disproportionality terms derives from computing the proportion of reports, which include the given event using data from




all non-target drugs. The larger the number of non-target drugs and non-target reports in the database, the better the chance the “expected” proportion is a reasonable basis for comparison. A simple decision rule is generally invoked to determine if a signal of disproportionate reporting is present, after comparing the target proportion to an expected proportion. It is important to note that this “signal” is statistical or numeric and still requires review before being considered a safety signal in the regulatory context. For example, if the ratio of the proportions exceeds some pre-determined threshold (e.g., the computed value exceeds 2.0), the target drug-event pair are considered for further examination in the SM process. This analysis is repeated for all drug event pairs and all drugs of interests, often uniformly without consideration of factors, such as time on the market, the treated patient population, or impacts of events that might stimulate spontaneous reporting. Various publications have described the computational specifics for frequentist and Bayesian approaches to disproportionality.14, 15, 16

The appropriateness of the use of disproportionality analyses depends upon the composition of the databases. Generally, if databases are sufficiently large and sufficiently diverse, these methods are appropriate.14, 15, 16 However, many company-owned databases contain data for a small number of products making disproportionality analyses inappropriate. Others have proposed alternatives based on the frequency of collected reports, including change-point analysis, sieve analysis, and combinations of frequency and disproportionality methods.16, 17 What is “expected” in the case of frequency analyses is derived from the trend over time of incoming data for a given drug and event. As these methods focus on the time trend of a drug-event pair over time, they are not dependent upon other drugs and events within the database. However, these methods are impacted by external factors that may stimulate reporting, such as publicity, product marketing efforts, or increasing product sales and distribution. Statistical process control methodologies made popular by manufacturers in the 1920s and 1930s are one way of attempting to control for external factors (e.g., batch size) while identifying what is different than expected based on historical experience.

Considerations for Refining Methods and Adjusting Thresholds Regardless of the database or analysis method, one cannot assume that the initial assumptions of analysis and threshold for determination of a signal of disproportionate reporting (or similar in frequency-based methods) perform optimally for the defined performance criteria (e.g., earliest detection, fewest false positives, positive predictive value with adequate specificity and sensitivity). Best Practice Recommendation: There should be a process for annual assessment that allows a company to adjust thresholds, change analysis methods, or some combination to reduce false positives and increase the overall positive predictive value (PPV) of the signal detection approach without impairing sensitivity. The purpose of the annual evaluation of a signal detection system is to determine how accurately it can detect quantitative changes in a spontaneous report system. Some signals are detected by a single case or generated through a regulatory query and not necessarily expected to be predicted by quantitative approaches to the detection stage. The quantitative system’s performance should be considered after removing these non-quantitative signal sources.




A false negative occurs if the system fails to identify a signal which should have been identified through quantitative methods. Generally, in PV, false negatives are viewed as the worst possible occurrence. Therefore, most systems are designed to guard against the possibility of “missing something.” This safeguard will necessarily increase the number of false-positive safety observations that can incorrectly identify risks, influence the benefit-risk balance (possibly discouraging patient use), and unnecessarily increase costs (e.g., more resources needed, less efficiency). In this operational context, the effort expended in the evaluation of false positives is non-value adding; effort then could be used to address other clinically valuable actions within a PV system.

The Operating Characteristics of a Statistical Signal Detection Analysis Determination of the performance of signal detection analyses is routinely completed through the calculation of four statistics: PPV, negative predictive value (NPV), sensitivity, and specificity. Taken together, they describe the overall ability of the analysis (or set of analyses) to correctly identify statistical signals while minimizing errors (false positives and false negatives). Specifics for calculating these statistics are available.18

Summarizing Performance and Updating Analyses When selecting or changing thresholds or adding/removing requirements to the signal detection algorithms, one should consider and assess the four key measures (PPV, NPV, sensitivity, and specificity) relative to the risk of changing these measures. A typical challenge in the pharmaceutical industry is the improvement of the signal-to-noise ratio (essentially PPV) so that scarce resources, such as physicians and scientists can focus on those DECs that are more likely to result in true signals and actionable safety outcomes. This improved PPV can be achieved through increasing thresholds and supplementing the signaling method (e.g., disproportionality) with other requirements, such as minimum required case counts, or a minimum number of cases with “sufficient quality” which can be measured by methods akin to VigiGrade.19 The guaranteed increase in PPV comes with a potential penalty for sensitivity. The key to assessing the extent of this penalty is to retrospectively apply the new, more specific algorithm to a dataset of previously identified signals and subsequently assessing the number of false negatives in that set. This approach can be applied if the MAH maintains a well-defined set of signals and can recreate the results from new algorithms as if they were applied to the data at the moment of the original signal detection. The extent of the loss of sensitivity must then be assessed in the context of the risk presented, and by the potential to detect these false negatives through other means, such as simple line listings or single cases inspection. It has been reported that conducting analyses on subgroups of patients based upon therapeutic area, patient demographics, or AE qualities may positively alter the sensitivity and precision of the signal detection system.20 That is, applying disproportionality analysis within subgroups can result in marginal gains in both sensitivity and precision. The optimal balance and trade-offs of these considerations must be weighed by drug safety experts as the “right choice” will be different for different companies, depending on the nature and lifecycle of their products and regulators. Some approaches may accept more risk (e.g., risk-based approaches) to permit limited resources to address the most significant public health concerns. The result of this modeling of alternatives should be documented in a formal report to support any corresponding changes to signal detection analyses.




External Databases Because each database has slightly different characteristics for reporting and analysis, which may result in differential value, our survey asked respondents to summarize their experience with publicly available databases and other data sources. Table 2 presents signal triggers from a period of “stable signal detection methodology” where methods and processes for signal detection did not undergo significant changes.

Table 2. Summary of Industry Survey Results: Signals and Primary Source of Information

TRIGGER # SIGNALS # VALIDATED

SIGNALS # CONFIRMED

SIGNALS % OF SIGNALS CONFIRMED

Single case assessment 501 138 30 5.98%

Aggregate AE reports (spontaneous & solicited)

648 277 93 14.35%

Periodic reports (e.g., PSUR)

80 64 12 15.00%

Literature/presentation 555 264 57 10.27%

Health authority 1233 1115 386 31.31%

Clinical trial 195 159 72 36.92%

Observational data 31 3 0 0.00%

Product quality 242 146 15 6.20%

Audit/inspection 36 5 1 2.78%

Other 446 269 99 22.20% **AE = adverse event; PSUR = Periodic Safety Update Report

As Table 2 demonstrates, the percentage of signals which are eventually confirmed vary considerably across the sources of safety data, ranging from 0% to 31.3%. These unpublished TransCelerate survey results from 2018 also indicate that resource is spent identifying, managing, and evaluating signals across numerous data sources, while two data sources, clinical trials and health authorities, yield 68% of confirmed signals. The term “confirmed signal” in this process map corresponds to a new/changed risk according to the GVP Module IX. Similarly, previous unpublished work conducted by a TransCelerate workgroup in 2018 indicates that over a three-year period of “stable signal detection methodology” (i.e., where methods and processes for signal detection did not undergo significant changes), company-owned spontaneous databases triggered numerous investigations. However, very few safety actions (e.g., label changes) were taken. The resources consumed to mine databases and evaluate signals appear to be out-of-step with the insights generated. A value-based approach should be considered to apply resources more efficiently. Survey results further indicated the degree to which data sources overlap. Respondents were asked to report their experience analyzing eRMR data from EVDAS and whether the insights generated were novel. For 10 out of 10 responding companies, results found in eRMR were consistent with an analysis of company-owned sponsor databases.




This same workgroup undertook a separate analysis of the same drug products across VigiBase®, EVDAS, and FAERS to compare the results.21 A standardized analytical approach and fixed threshold criteria were applied to ensure consistency of identification of signals of disproportionate reporting within each database. Results indicate that >90% of identified signals of disproportionate reporting (SDRs) were present in the other databases as well. These results confirm the widely held belief that leveraging different publicly available data sources produces few unique findings.

Triage The triage step can be defined as the review and application of clinical context/medical judgment of a safety observation to determine whether the safety observation merits further verificatory action (i.e., should be considered a safety signal). In some companies, triage/validation are combined in one process step. For others, this is a formal 2-step process with distinct outputs. For the sake of avoiding repetition, we have included triage within the description of signal evaluation.

EVALUATION STAGE Even for those companies where no formal distinction exists between triage and validation, a majority of safety observations are assessed and dismissed rapidly, with a small remainder going through a longer process. In essence, all companies have a process that allows for a continuum of immediate assessment to an inspection of readily available data before a safety observation is declared a validated signal. Some companies observe a formal distinction between triage and validation, while others operate without that distinction. There are certain pros and cons to each process approach as outlined in Table 3.




Table 3. Evaluation Stage: Distinct Triage and Validation versus Integrated Validation

FEATURE DISTINCT TRIAGE/

VALIDATION INTEGRATED VALIDATION

Tracking and Documentation Safety observations that are dismissed are not signals and do not have to be tracked to the same extent as signals.

Everything is a signal and needs to be tracked.

Consistency of Application of Criteria

Signals can be dismissed quickly based on pre-defined criteria.

Risk of lack of consistency for early dismissed signals as there is a continuum of criteria

Amount of Work Needed for a Signal

Large numbers of safety observations can be assessed very rapidly (assuming the formal triage step is clearly defined).

Risk of spending too much time on average on a safety observation

Roles and Responsibilities Ability to split responsibilities (e.g., one group does triage another the validation)

Less flexibility in terms of roles and responsibilities

Flexibility Less flexibility: not every safety observation undergoes the same scrutiny as the triage step is usually narrowly defined

The integrated process represents more of a continuum and thus, allows more flexibility in what is done to inspect a signal – a tailored approach is possible.

Validation Signal Validation: the process of evaluating the data supporting the detected signal in order to verify that the available documentation contains sufficient evidence demonstrating the existence of a new potentially causal association or a new aspect of a known association, and therefore justifies further analysis of the signal.1 Validation is not a process of confirming a safety issue. It is the preliminary assessment of relevant available evidence for a signal to clarify whether a more thorough review is necessary or further action is needed. The review may strengthen a potential safety signal and indicate the need for further assessment. It may also suggest that further assessment is not needed based on the safety data reviewed. The aim of the review is not to determine if an event has a causal association with a product (i.e., ADR assessment).

Signal Assessment A “signal” is information arising from one or multiple sources, including observations and experiments, which suggests a new potentially causal association, or a new aspect of a known association between an intervention and an event or set of related events, either adverse or beneficial, that is judged to be of sufficient likelihood to justify verification.22 Called either signal assessment (GVP) or signal evaluation, the objective of this activity is to further evaluate a validated safety signal to identify the need for additional data collection or any regulatory action. It consists of an assessment of the available pharmacological, non-clinical and clinical data and information from other sources. This review should be as complete as possible




regarding the sources of information, including the application dossier, literature articles, spontaneous reports, expert consultation, and information held by MAHs and competent authorities. The goal of signal assessment is to confirm whether a safety signal is a confirmed safety issue (e.g., ADR) and the outcome of the signal assessment is the final assessment of the safety signal (i.e., the confirmation or non-confirmation of a safety issue). Signal assessment results in a recommendation that either no further action is required at this point or further action is needed. Survey results indicate that most surveyed member companies rely primarily on medical judgment and expert knowledge for signal assessment. Specific qualitative and quantitative methods may be used for individual case assessment. Still, few companies use standard algorithms for determining if there is a causal relationship between an AE and the use of a product, recognizing that the approach to assessing a signal depends on the nature of the signal and the data available. Generally, a senior safety staff member or a senior multidisciplinary team consider all evidence and make a qualitative decision. A review of all available sources of data is essential at the assessment stage of the SM process. Some companies use a template for signal assessment to promote consistency and ensure consideration of all applicable data sources. Organizations may use systematic approaches for reviewing large numbers of cases (e.g., identifying cases with positive dechallenge or rechallenge and reviewing additional cases only if the assessment remains inconclusive). Some companies utilize various visualization tools, but the utility and impact of such tools on assessment and decision making are not clear. Therefore, two areas to explore further are the following:

• The current impact of visualization tools in decision making. If this is not determined to be impactful, explore tools that may be impactful.

• As decision makers consider quantitative approaches to benefit-risk assessment, they may explore if a quantitative approach may be used to complement the signal assessment. Can they, for example, possibly weight the different data sources considered in validation and assessment to provide an overall weight for consideration?

ACTION STAGE

Actionable Safety Outcomes Assessment of a safety signal concludes with an outcome that reflects the evidence supporting the likelihood of a causal association between a product and event. That outcome drives subsequent actions as shown in Table 4.




Table 4. Actionable Safety Outcomes

OUTCOME OF SIGNAL ASSESSMENT

ACTIONS TO CONSIDER

Evidence Supportive of a Causal Association

• Update product labeling (e.g., new ADR, new warning or precaution)

• Amend clinical development plans/ongoing studies

• Communicate new safety information directly to HCPs/patients

• Implement additional risk minimization measures (e.g., educational materials, restricted distribution plan)

• Implement enhanced PV measures (e.g., targeted follow-up of AE reports)

Evidence Inconclusive • Generate additional data (e.g., new studies/new analyses)

• Implement enhanced PV measures (e.g., more frequent signal detection, review of additional data sources, targeted case review)

• Re-assess the signal in a defined timeframe

Evidence Not Supportive of a Causal Association

• Continue routine PV

**ADR = adverse drug reaction; AE = adverse event; HCP = healthcare professional; PV = pharmacovigilance

Cross-functional input and expertise are often required to determine and execute appropriate actions in response to the outcome of a signal assessment. Best practices for actionable safety outcomes include following standard operating procedures that clearly describe:

• Escalation of issues internally and to partners and regulatory authorities

• Accountabilities for making, communicating, and documenting decisions

• Responsibilities for prioritizing and executing actions

• Framework for prioritizing actions based on the potential impact of the new safety information on public health or the benefit-risk profile of the product

• Standard timeframes for executing actions based on prioritization PV and regulatory quality management systems should be in place to monitor the execution of actions, ensuring that the right information reaches the right audience at the right time. Decision makers should evaluate the effectiveness of these actions in communicating and minimizing risk for safety issues with a significant impact on public health or the benefit-risk profile of a product.

Documentation and Tracking Across the entire SM process, GVP Module IX places an unequivocal emphasis on monitoring and documentation of all related activities and their outcomes.1 As for any critical process, the MAH is obliged to document the process itself and should describe it in a standard operating procedure or the PV system master file (PSMF). This process documentation, in the PSMF or other standard operational process documentation, must include the monitoring frequency (including any changes) of products in the scope of the SM process and justification for the choice of monitoring frequency. It makes good sense for companies to adopt clear and unambiguous description of the MAH’s signal detection criteria and rationale for the selection of




methods. We further propose using a matrix or flow-diagram to tabulate complex criteria and thereby aid understanding. We also propose documenting product-level surveillance literature searches with version control (e.g., dates when companies employed each version of the search strategy). In terms of decisions made for each signal, we suggest tracking and documenting all stages of the signal’s “life-cycle” to the extent necessary to justify a given decision at each stage of the process. The quantity and level of detail of documentation increase as the signal develops through the process. For example, it may be enough to use predefined or standard annotations during triage of safety observations (e.g., Event is anticipated with the background disease for the indicated population.) Whereas one may expect a full evaluation of all available safety data during a formal safety review assessment from which an actional safety outcome may result. In any case, traceability is essential at every stage. Table 5 summarizes the proposed minimum items for documentation at key stages in the SM process.

Table 5. Proposed Minimum Items for Documentation Throughout the SM Process Item

ITEM FOR DOCUMENTATION

*COULD BE COMBINED* ASSESSMENT

DETECTION TRIAGE &

VALIDATION

Drug-Event Combination (Generally at PT Level) X

X X

Publication Citation (Literature Surveillance) X

Safety Topic Name (Including Relevant MedDRA Concepts)

X X

Reference Safety Information Designated Medical Event Targeted Medical Event (Or Decision to Keep Under Review)

X X X

Any Previous History of Signal or Topic

X X

Count of Case Reports in the Review Period with Relevant Breakdown Counts

X X

Disproportionality Score X

X X

Source of Signal

X X

Reason for Safety Observation/Signal X

X X

Date Detected X

X X

Prioritization X

X X

Date of Notification to Health Authorities (If Applicable)

X X

Date of Review X

X X

ID of Person(s) Reviewing Safety Observation/Signal X

X X

Associated Drug-Event Combinations Also Reviewed X

X X




Other Data Sources Considered in the Review

X X

Whether or Not Signal Validated

X X

Reason for Refutation

X X

Current Status of Signal (Open or Closed) X

X X

Date Signal Closed

X X

Disposition of Safety Observations (e.g., Continue to Evaluate/Close/Complete)

X X

Recommendations for Further Action

X

Classified as Potential or Identified Risks, and Whether Important or Not

X

Evaluation Report (Including a Benefit-Risk Reassessment)

X

Links to Supporting Documentation (e.g., Meeting Minutes, Reports)

X X X

Full-Text Article for Signals Arising from Literature Surveillance (If Available)

X X

**ID = identification; MedDRA = Medical Dictionary for Regulatory Activities; PT = preferred term; SM = signal management; ‘X’ indicates proposed documentation at key stages in the signal management process

Although standard document management systems and collaboration tools can manage a large proportion of the documentation required for performing SM activities, these tools should include audit functions and version control capabilities. Specialized SM tools are available that track actions executed in process-driven workflows and include a non-modifiable audit trail compliant with applicable Good Clinical Practices (GCP) standards for computerized systems. Such systems generally permit upload and storage of supporting documents created in common formats (e.g., spreadsheets, documents, PDFs). Companies should document the training of staff involved in the SM process and retain this for audit or inspection purposes. For staff involved in reviewing EudraVigilance outputs, there is an expectation of familiarity with the specific guidance and training materials made available online by the European Medicines Agency (EMA). In practice, this means keeping records of both the content and delivery of the training.

DISCUSSION The introduction of GVP Module IX in June 2012 effectively mandated the implementation of formal routine SM for MAHs, and the 2017 revision introduced the requirement to monitor the EVDAS database.1 Additional data sources for the detection and evaluation stages have become available over the past decade, together with the development of new signal detection and analysis methods. IMI PROTECT showed that the performance of disproportionality methods is highly dependent on the actual database, such that a combination of thresholds/method has high sensitivity and precision in one database and relatively low performance in a different one. The conclusion is that these methods/threshold combinations should be optimized for the




spontaneous reporting database in question, ideally with a reference dataset of positive and negative controls derived from that same database.2 Crucially, in the absence of a hierarchy of evidence to support SM practices agreed upon among pharmacovigilance professionals, it is not clear whether widening the net in search of safety signals promotes the public health or merely detracts from efforts to attend to the most pressing safety concerns. This paper described at least some aspects of this question and proposed some ideas on ways to address other aspects. We found that attempts to clarify and harmonize definitions of concepts such as signal, evaluation, and validated signal inevitably descend into semantic arguments that may not support the practical implementation of a workable SM system or efficient collaboration between MAHs and regulators. This paper identified a simplified framework for describing end to end SM that captures the core activities we believe most MAHs undertake while allowing for the variance in definitions and procedural implementation across organizations. Because there is no model or standard blueprint or detailed requirements list for implementing a detection stage framework, the following considerations may help inform the establishment of new processes or re-appraisal of an existing one (Table 6).

Table 6. Consideration for Establishing or Re-appraising a Signal Management Process

CONSIDERATION ADDITIONAL CONSIDERATIONS

RESOURCES Which sources of cases should be included in statistical signal detection?

The assumption underpinning disproportionality analysis is the independence of each ICSRs within the database. Therefore, in general, only spontaneous cases are included.

Is masking likely to be a problem? If so, how can we compensate for it?

Masking can occur in non-heterogenous databases (e.g., company databases with a limited portfolio of products spanning a narrow range of therapeutic areas) or for large numbers of solicited reports (e.g., from lawyers undertaking a class action lawsuit). Maignen et al. proposed methods to identify masking (i.e., need definition), and these may be useful where local knowledge of database quality is lacking.23, 24 Where significant masking is seen to exist, it may be expedient to exclude the relevant ICSRs from analyses. However, the rationale for excluding any cohort of ICSRs should be justified and thoroughly documented and an alternative means made available to permit their analysis.

Which database(s) should be used if considering external databases for signal detection or signal strengthening?

The TransCelerate survey and study of Jokinen et al. suggest that new safety insights gained from conducting signal detection in safety data sources other than company safety databases are relatively few.25 A recent study by Vogel et al. indicates a significant overlap in SDRs exists between EV, FAERS, and VigiBase® for many products. However, the degree of overlap is influenced by the threshold (i.e., level of MedDRA hierarchy used).21

ACTIVITIES Is a statistical approach to signal detection necessary or

• Size and heterogeneity of database

• Number of products of interest in the portfolio




appropriate? If yes, use company safety or external database?

• Degree of representation of products of interest in the target data sources and extent of SDR overlap between sources

If using statistical signal detection approaches, which method is best?

There is little to choose from between the methods in terms of overall signal detection performance as measured by sensitivity and PPV. However, Bayesian methods are computationally more demanding than frequentist methods, and the derivation may be harder for the non-specialist to grasp.

What is the correct choice of signal detection criteria to apply if using statistical approaches?

There is a direct trade-off between sensitivity and specificity when using statistical methods, and performance is likely to change over time. It is essential to establish a baseline level of detection stage performance to assess the impact of changes to the detection criteria, or if considering completely new approaches.

Should the same signal detection criteria be applied to all products?

Statistical signal detection performance appears to deteriorate over time4 therefore, adopting different signaling criteria for older versus newly marketed products may be justifiable (i.e., favor lower sensitivity and greater PPV for mature products with a well-established safety profile, while maintaining higher sensitivity [and consequently lower PPV] for recently marketed products).

Should stratification or subgrouping be employed in quantitative signal detection?

Following the study of Seabroke et al., there may be a benefit of conducting the detection stage for specific subgroups.20 However, the impact of dividing sparse data into even fewer subsets needs to be considered and may only be appropriate when using large external databases.

**EV = EudraVigilance; FAERS = FDA Adverse Event Reporting System; ICSR = Individual Case Safety Reports; MedDRA = Medical Dictionary for Regulatory Activities; PPV = positive predictive value; SDR = signals of disproportionate reporting

This paper considers SM in three distinct stages: detection, evaluation, and action. The resultant outputs of each stage are called safety observations, signals, and actionable safety outcomes, respectively. In practice, this progression is accompanied by evidence gathered from multiple data sources. However, in principle, a single index case report observed at the detection triage phase could be sufficient to precipitate a change to the reference safety information for a product, the intermediate phases of signal validation and assessment becoming largely redundant and called out mainly to fulfill procedural requirements. Putting aside such rare occurrences, however, it is worth noting that the transition through triage demarks the boundary between hypothesis generation and hypothesis testing (i.e., signal generation and causality assessment). Yet, that change is rarely reflected or even acknowledged in the approach to signal assessment employed across the discipline. We propose a process for regular performance assessment since detection stage performance can deteriorate over time. Key performance indicators might include sensitivity and positive predictive value and an index of time to signal (i.e., the time from first market authorization to when the safety observation first occurs). We have identified meaningful and comparable points within the SM process for measuring detection stage performance, including (1) the number of safety observations converted into signals following the triage phase and (2) the number of numeric signals turned into actionable safety outcomes at the assessment phase. The overall approach could be individually applied for sources of safety signals, as well as in aggregate, to discern the relative contribution of each source to overall detection stage performance.




A suitable armamentarium of analytic methods, including statistical approaches and visualization tools, are needed to address the question of causality head-on, rather than merely collecting and critiquing accessible information. Although we are unable to propose specific methods at this time, it is worth calling out the need for such methods to be developed and tested in real-life situations, and TransCelerate and other collaborative projects have the potential to act as catalysts in this endeavor. Another area we posit for further research and development is “risk-based” or “value-based” SM. Risk-based assessment approaches are not uncommon in many industries, including pharmaceutical development and production. In the domain of SM, this means applying higher-order selection criteria to those already applied during the detection stage. It also implies an approach where we maximize value but do not handle all safety signals equally. It arguably may be best, for example, to prioritize the assessment of signals from newly marketed products over mature products with a long history of exposure. We encourage industry stakeholders to explore this argument for empirical research (e.g., pilot studies) while considering our SM process and proposals. Such studies are needed to validate whether or not we can simultaneously improve signal quality and detection efficiency while demonstrating real-world value to patients and regulators.

CONCLUSION As initially stated, we sought to identify new data sources and novel methodologies that could drive the advancement of signal detection and management. What we identified, however, was the need for a more fundamental decision-making process in SM to identify the optimal fit among existing sources and methods given the backdrop of an increasing supply of data and demand for quality. We propose that this is how safety analytics can be most readily advanced. By working within such a framework for SM, the articulation of best practices can be facilitated. A widely accepted framework better ensures that all stakeholders are referring to the same concepts, decision points, and process. This brings efficiency and eases interactions between MAHs, agencies, and business partners.




REFERENCES

1. European Medicines Agency (EMA) and Heads of Medicines Agencies. Guideline on Good Pharmacovigilance Practices: Module IX—signal management [EMA/827661/2011]. 2017. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-gvp-module-ix-signal-management-rev-1_en.pdf. Accessed 23 Oct 2019.

2. Wisniewski AF, Bate A, Bousquet C, Brueckner A, Candore G, Juhlin K, et al. Good signal detection practices: evidence from IMI PROTECT. Drug Saf. 2016;39(6):469–490.

3. Hill R, Hopstadius J, Lerch M, Noren GN. An attempt to expedite signal detection by grouping related adverse reaction terms. Drug Saf. 2013;35(12):1194-1195.

4. European Medicines Agency. EudraVigilance. 2019. http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/general/general_content_000686.jsp&mid=WC0b01ac0580a69261. Accessed 20 Mar 2018.

5. European Medicines Agency (EMA) and Heads of Medicines Agencies. Guideline on Good Pharmacovigilance Practices: Module VI—collection, management, and submission of reports of suspected adverse reactions to medicinal products [EMA/873138/2011 Rev 2]. 2017. https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/guideline-good-pharmacovigilance-practices-gvp-module-vi-collection-management-submission-reports_en.pdf. Accessed 31 Mar 2020.

6. European Medicines Agency (EMA) and Heads of Medicines Agencies. Guideline on Good Pharmacovigilance Practices: Module VII—periodic safety update report [EMA/816292/2011 Rev 1]. 2013. https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-gvp-module-vii-periodic-safety-update-report_en.pdf. Accessed 31 Mar 2020.

7. U.S. Food and Drug Administration (FDA). Code of Federal Regulations Title 21: Postmarketing reporting of adverse drug experiences [FDA/21CFR314.80]. 2019. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=314.80. Accessed 31 Mar 2020.

8. Chan KA, Hauben M. Signal detection in pharmacovigilance: empirical evaluation of data mining tools. Pharmacoepidemiol Drug Saf. 2005;14(9):597-599.

9. Harpaz R, Callahan A, Tamang S, Low Y, Odgers D, Finlayson S, et al. Text mining for adverse drug events: the promise, challenges, and state of the art. Drug Saf. 2014;37(10):777–790.

10. Christensson C, Gipson G, Thomas T, Weatherall J. Text analytics for surveillance (TAS): an interactive environment for safety literature review. Drug Inf J. 2012;46(1):115-123.

11. Caster O, Dietrich J, Kurzinger ML, Lerch M, Maskell S, Noren GN et al. Assessment of the utility of social media for broad-ranging statistical signal detection in pharmacovigilance: results from the WEB-RADR project. Drug Saf. 2018;41(12):1355-1369. doi:10.1007/s40264-018-0699-2

12. van Stekelenborg, J., Ellenius, J., Maskell, S. et al. Recommendations for the use of social media in pharmacovigilance: lessons from IMI WEB-RADR. Drug Saf. 2019;42(12):1393–1407. https://doi.org/10.1007/s40264-019-00858-7

https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-gvp-module-ix-signal-management-rev-1_en.pdf.



http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/general/general_content_000686.jsp&mid=WC0b01ac0580a69261

http://www.ema.europa.eu/ema/index.jsp?curl=pages/regulation/general/general_content_000686.jsp&mid=WC0b01ac0580a69261

https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/guideline-good-pharmacovigilance-practices-gvp-module-vi-collection-management-submission-reports_en.pdf



https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-gvp-module-vii-periodic-safety-update-report_en.pdf

https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-good-pharmacovigilance-practices-gvp-module-vii-periodic-safety-update-report_en.pdf

https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=314.80

https://doi.org/10.1007/s40264-019-00858-7




13. European Medicines Agency. Guideline on Good Pharmacovigilance Practices (GVP) Module IX Addendum I – Methodological Aspects of Signal Detection from Spontaneous Reports of Suspected Adverse Reactions. 2016 [updated 4 August 2016; cited 2016 22 September]. http://www.ema.europa.eu/docs/en_GB/document_library/Regulatory_and_procedural_guideline/2016/08/WC500211715.pdf

14. Quattrini G, Zambon A, Simoni L, Fiori G. PRM211: Disproportionality measures used in signal detection: an assessment on pharmacovigilance adverse event reporting system data. Value in Health. 2015;18(7):A720.

15. Jokinen JD, Lievano F, Scarazzini L, and Truffa M. An alternative to disproportionality: a frequency-based method for pharmacovigilance data mining. TIRS. 2018;52(3):294–299. https://doi.org/10.1177/2168479017728986

16. Candore G, Juhlin K, Manlik K, Thakrar B, Quarcoo N, Seabroke S, et al. Comparison of statistical signal detection methods within and across spontaneous reporting databases. Drug Saf. 2015;38(6):577–587.

17. Xu Z, Kass-Hout T, Anderson-Smits C, Gray G. Signal detection using change point analysis in postmarket surveillance. Pharmacoepidem Dr S. 2015;24(6):663-668. doi:10.1002/pds.3783.

18. Trevethan R. Sensitivity, specificity, and predictive values: foundations, pliabilities, and pitfalls in research and practice. Front Public Health. 2017;5:307. doi:10.3389/fpubh.2017.00307

19. Bergvall T, Niklas Norén G, Lindquist M. VigiGrade: A tool to identify well-documented individual case reports and highlight systematic data quality issues. Drug Saf. 2014;37(1):65-77. doi:10.1007/s40264-013-0131-x.

20. Seabroke S, Candore G, Juhlin K, Quarcoo N, Wisniewski A, Arani R, et al. Performance of stratified and subgrouped disproportionality analyses in spontaneous databases. Drug Saf. 2016;39(4):355 364.

21. Vogel U, van Stekelenborg J, Dreyfus B, et al. Investigating Overlap in Signals from EVDAS, FAERS, and VigiBase®. Drug Saf. 2020;43:351-362. https://doi.org/10.1007/s40264-019-00899-y.

22. Practical aspects of signal detection in pharmacovigilance, Geneva: Council for International Organizations of Medical Sciences (CIOMS), 2010.

23. Maignen F, Hauben M, Dogné JM. A mathematical framework to quantify the masking effect associated with the confidence intervals of measures of disproportionality. Ther Adv Drug Saf. 2017;8(7):231–244.

24. Maignen F, Hauben M, Hung E, Van Holle L, Dogne JM. Assessing the extent and impact of the masking effect of disproportionality analyses on two spontaneous reporting systems databases. Pharmacoepidem Dr S. 2014;23(2):195-207.

25. Jokinen J, Bertin D, Donzanti B, Hormbrey J, Simmons V, Li H, et al. Industry assessment of the contribution of patient support programs, market research programs, and social media to patient safety. TIRS. 2019;53(6):736-745.

http://www.ema.europa.eu/docs/en_GB/document_library/Regulatory_and_procedural_guideline/2016/08/WC500211715.pdf

http://www.ema.europa.eu/docs/en_GB/document_library/Regulatory_and_procedural_guideline/2016/08/WC500211715.pdf

https://doi.org/10.1177/2168479017728986

https://doi.org/10.1007/s40264-019-00899-y




Acknowledgements First, a big thank you to our team of authors—Antoni Wisniewski (AstraZeneca), Andres Gomez (Bristol-Myers Squibb Company), Jeremy Jokinen (AbbVie), Karol LaCroix (GlaxoSmithKline), Anju Garg (Sanofi), Neal Grabowski (Amgen), and Richard Hermann (AstraZeneca)—for all their hard work. We also want to acknowledge our dedicated group of contributing editors for their helpful reviews of the manuscript: John van Stekelenborg (Johnson & Johnson), Ulrich Vogel (Boehringer Ingelheim), and Benoît Vroman (UCB). Last, a special thanks to Michelle Thornton (formerly of Merck) for her early input and guidance on this manuscript creation. Many thanks to you all!

SIGNAL MANAGEMENT · implementation considerations within a generic safety signal management...

Documents

Transcript of SIGNAL MANAGEMENT · implementation considerations within a generic safety signal management...