Validation of Statistical Signal Detection Procedures in EudraVigilance Post-Authorization Data A Retrospective Evaluation of the Potential for Earlier Signalling

Background: Screening large databases of spontaneous case reports of possible adverse drug reactions (ADRs) is an established method of identifying hitherto unknown adverse effects of medicinal products; however, there is a lack of consensus concerning the value of formal statistical screening procedures in guiding such a process. This study was performed to clarify the nature of any added benefits and additional effort required when established pharmacovigilance techniques are supplemented with statistical screening. Objective: To evaluate whether statistical signal detection in spontaneous reporting data can lead to earlier detection of drug safety problems and to assess the additional regulatory work entailed. Methods: Using the EudraVigilance post-authorization module (EVPM), a screening procedure based on the proportional reporting ratio (PRR) was applied retrospectively to examine if regulatory investigations concerning ADRs in a predefined set of products could have been initiated earlier than occurred in practice. During the same time period, between September 2003 and March 2007, the number of PRR-based signals of disproportionate reporting (SDR) that arose in the same set of products was calculated and evaluated to determine the number requiring investigation. The outcome is expressed as the ratio of the number of SDRs requiring investigation compared with the number of signals pre-empted by the statistical screening approach. In those cases where the signal was discovered earlier, the delay was calculated between identification by the PRR method and by the method that originally identified the signal. Results: In 191 chemically different products, 532 adverse reactions were added to the summary of product characteristics during the study period. Of these, 405 were designated as important medical events (IMEs) based on a comprehensive predefined list. Of the IMEs, 217 (53.6%) were identified earlier by the statistical screening technique, 79 (19.6%) were detected after the date at which they were raised by standard pharmacovigilance methods and 109 (26.9%) were not signalled during the study period. 1561 SDRs requiring further evaluation were detected during the study period, giving a ratio of 7.2 assessments for each signal pre-empted. The mean delay between the discovery of signals using the statistical methods in the EVPM and established methods in the 217 cases detected earlier was 2.45 years. A review resulted in clear explanation for why the statistical method had not pre-empted detection in all but 77 of 188 cases. Conclusions: The form of statistical signal detection tested in this study can provide significant early warning in a large proportion of drug safety problems; however, it cannot detect all safety issues more quickly than other pharmacovigilance processes and hence it should be used in addition to, rather than as an alternative to, established methods.