The design of a health care product surveillance program should take into account product class effects and indications, the patient user and prescriber populations, and the route of administration. It should also be appreciated that use patterns may change over time, thereby necessitating alterations in the structure of the surveillance program itself. Venulet has published an example in which a product-specific surveillance scheme is designed and diagrammed (Venulet, 1988), while Klincewicz has pointed out the importance of linking product monitoring mechanisms with the periodic safety update report (PSUR) requirements found in the ICH E2C document (Klince-wicz et al., 1999). An exhaustive review has also been published recently that focuses on AE signalling methods and AE surveillance program design (Clark et al., 2001).
In selecting sorting methodologies, it is noteworthy that the expedited (serious + unlabelled) report sorting procedure and several PSUR line listings are mandated by regulation, and should therefore be included in essentially all such designs. Sorting methods based on designated medical events would also appear to be appropriate for many spontaneous report data sets, given the regular appearance of a selected list of AE types throughout the history of health care product monitoring. For products that give rise to small numbers of reports, one or two sorting methods will usually be sufficient to address report-based monitoring needs.
The use of automated identification procedures would logically be reserved for larger and more complex report databases. In creating report screening strategies, it is helpful to conceptualize the identification step as the use of screening tests to select "interesting" product-AE pairs for further study. In this view, the use of effective identification procedures for product monitoring resembles the use of screening tests for other public health applications, and implies special attention to false positive rates. The false positive rate for an AE surveillance program can be decreased by selecting identification procedures with a higher specificity (and, therefore, a lower sensitivity), while its false negative rate can be decreased by using multiple identification procedures, each of which operates on a different principle, and by repeat screening on a regular basis. The overall effectiveness of the report phase of an AE surveillance program can be evaluated by calculating the report-based positive predictive value, which is equivalent to the proportion of signal evaluations found at the case-based phase to be "interesting". A lower than desired predictive value suggests that the specificity of the identification methods should be increased.
Experience suggests that, despite past difficulties, there remains a place for both rule-based imputation screening and serial signalling methods in spontaneous AE surveillance programs, but that the use of both of these identification methodologies must be scrutinized carefully. Statistical models are unlikely to be helpful on a widespread basis at the identification step, because such tools are too complex and individualized to be effective as screening tests. Although not employed systematically in the past, screening for temporo-geogra-phical clusters may be effective for certain types of safety outcomes, especially those that are related to product preparation and local circumstances of use.
At present, inter-product identification methods appear to be promising developments for public-health-oriented databases, but would usually not be feasible for manufacturer-specific databases which, in comparison with other product-AE monitoring databases, are relatively small and highly selective.
In constructing case series, evaluators should clearly specify case definitions, and adapt them to the limitations of spontaneous data. When comparing such experience with expectation, the standardized reporting ratio and its modifications remains one of the most valuable tools currently available to program designers. This methodologic approach requires extensive knowledge of background incidence rates that is best confirmed empirically through literature reviews and/or database research. Many program administrators will want to have these resources and pertinent expertise available on an ongoing basis. Product monitors will likely want to regard inter-product comparisons of spontaneously based case series with care. Because of the many biases affecting interpretation, these methods are probably best thought of as a group of public-health-motivated analyses of last resort that are undertaken when better data cannot be acquired in an appropriate time frame.
A review of the literature pertaining to case series characterization methods suggests that, when carried out thoughtfully and represented appropriately, such data can be helpful to prescribers. In formulating characterization analyses, the possibility of a relationship between the characterized feature and the adverse event rate or other endpoint should always be formally considered. In presenting such data, the analyst should also bear in mind the possibility that the characteristics of the reported case series may be different from population-based case series, and commentary that addresses this uncertainty should be provided.
Was this article helpful?