Imputation screening involves the use of imputation-based procedures to find "interesting" pro-duct-AEs in an AE report database. Imputation screening procedures have four basic attributes: (1) application on a report-by-report basis, (2) amenability to automation or semi-automation, (3) sensitivity and specificity for finding "interesting" product-AEs, and (4) focus on finding reports possessing summary causality testing that favors the monitored product (i.e. a summary likelihood ratio greater than one) (see Table 18.3, Part A). Those imputation screening procedures that are selected for actual use should exhibit the features of a good screening test as they apply to the imputation screening environment, namely: (1) easy to use, (2) generalizable to many product-AE pairs, (3) capable of producing reproducible (uniform, reliable) output, and 4) associated with a low false positive rate (see Table 8.3, Part B). In the presence of a declining or low prevalence of
Table 18.3. Features of database imputation screening instruments.
A. Basic features
• Applied on a report-by-report basis
• Amenable to automation or semi-automation
• Associated with a sensitivity and specificity for finding ''interesting'' product-AE pairs
• Finds reports for which positive tests for causality have been reported
B. Desirable features
• Generalizable to many product-AE pairs
• Capable of producing uniform output
• Associated with a low false positive rate
"interesting" product-AE pairs, imputation screening can exhibit a high false positive rate. This circumstance leads to a low efficiency rating, and is inconsistent with the operational realities of AE surveillance programs.
For the past four decades, imputation methods have undergone continuous change, much of which has focused on individualizing them for use in medical diagnosis (Lane et al., 1987; Stephens, 1999). To the degree that this was successful, imputation instruments became more applicable for diagnosis, but less applicable as screening methods. The first imputation procedures devised in the 1960s depended entirely on subjective judgments made by physicians or other health care professionals (MacDonald and MacKay, 1964). In the 1970s, this evolved into formalized procedures based on rules in which assessor subjectivity was applied more uniformly through the use of algorithms or questionnaires (Irey, 1976a, 1976b). In the early 1980s, statistical modelling was first applied to imputation problems when Bayesian methodology was introduced (Lane, 1984; Auriche, 1985), while, in the late 1980s, AE-specific approaches were first proposed (Benichou and Celigny, 1991). Since that time, more complex statistical procedures, such as decision support algorithms, have also been suggested (Hoskins and Manning, 1992). Of these five groups of published methods by which single report imputation screening could potentially be performed (subjective, generalized rule-based, Bayesian odds model, AE-specific rule-based, and non-Bayesian statistical models), practical limitations have led to the widespread use of subjective and generalized rule-based methodology only. AE-specific rule-based methods and Baye-sian and non-Bayesian statistical models have had essentially no systematic use as screening devices in databases comprised of individual patient reports.
Three important functional issues have been raised about the applicability of single report imputation methods to AE data: (1) the effect of missing, unobtainable, or erroneous data elements (Irey, 1976, 1976b; Venulet, 1986; Meyboom and Royer, 1992); (2) the isolation of a single product out of a list of multiple potential causes (Mey-boom and Royer, 1992); and (3) the value of imputation in generating signals of previously undescribed product-AE pairs (Meyboom, 1998: Stephens, 1999). The first issue, data integrity, is especially pertinent to report-by-report assessments because individual patient report data are often incomplete or inaccurate (Edwards et al., 1990; Funch et al., 1993; Sicker et al., 1998). Additionally, many reports are not useful sources for contributory data elements because the pro-duct-AE pair is not amenable to classical imputation procedures (e.g. dechallenge evaluation in the presence of a non-resolving AE) (Stephens, 1999). The second issue, selection of a single product for monitoring focus in the presence of multiple potential product contributors, is frequently encountered in individual patient reports (Kramer, 1986; Meyboom and Royer, 1992). In general, imputation methods tend to have less value as the complexity of either the disease or treatment background increases. The third issue, initial discovery value, arises because operational causality assessments always depend to some degree on expectations shaped by prior experience. However, it has been well documented that important new spontaneous signals can originate from settings where prior experience is limited or misleading, thereby resulting in a delay in signal recognition (Inman, 1993).
IMPUTATION SCREENING (CAUSALITY) ASSESSMENTS
Subjective imputation (also called "global introspection'' (Kramer, 1986), "unstructured clinical judgment'' (Jones, 1994), or "striking case method'' (Amery, 1999)), involves the assignment of a causal rating to an individual spontaneous AE report based on medical diagnostic experience (MacDonald and MacKay, 1964). Subjective assessments have usually involved classification into multiple categories, such as the designations "documented", "probable", "possible", and "doubtful" that were first proposed in the 1960s and are still used in modified form in AE evaluations today (Cluff et al., 1964; Seidl et al., 1965). Subjective judgment probably remains the most widely used method by which imputation screening is performed in AE surveillance programs (Meyboom and Royer, 1992; Jones, 1994; Hartmann et al., 1997; Amery, 1999).
Subjectively generated imputation assessments have been shown to be associated with high levels of intra- and inter-rater variability (Karch et al., 1976; Koch-Weser and Greenblatt, 1976; KochWeser et al., 1977; Blanc et al., 1979; Naranjo et al., 1981), and can produce results that differ substantially from results derived using rules (Miremont et al., 1994). This imprecision appears to be largely related to the many factors that affect any particular AE occurrence, making reproduci-bility (i.e. uniformity of the result with repetition) difficult to achieve in the absence of an algorithm (Kramer, 1986; Jones, 1994). It is interesting to note, however, that subjective assessments performed by assessors with longstanding professional relationships are not necessarily less uniform than those based on formalized procedures (Grohmann et al., 1985). Such examples probably reflect the tacit adoption and systematic use of non-written rules by a group of evaluators.
Generalized rule-based methods (also called standardized assessment methods (Hutchinson et al., 1983; Hutchinson, 1986) or standardized decision aids (Naranjo, 1986)), involve rating a product-AE occurrence using a questionnaire or algorithm (Lane et al., 1987; Hutchinson and Lane, 1989; Jones, 1994; Meyboom et al., 1997b). Over the past three decades, a large number of such instruments have been published (Irey, 1976a, 1976b; Karch and Lasagna, 1977; Kramer et al., 1979; Naranjo et al., 1981; Stephens, 1999). All rule-based methods contain three basic components that have been designed by an expert (see Figure 18.4): (1) a set of structured responses to questions, (2) a weighting algorithm that translates question-specific values into a summary value, and (3) a scaling algorithm that equates summary value ranges to imputation ratings (probable, possible, etc.) (Hutchinson, 1986). Evidence has been published that rule-based methods can reduce intra- and inter-rater variability as compared with subjective judgment, thereby
Bayesian Odds Model
Tests For Causality And Epidemiologic Data
Questions Create Categorized Answers
Observer Inputs Responses Only
Weighting Algorithm Translates Question Specific Values Into A Summary Value
Bayesian Odds Model
Questions Create Categorized Answers
Was this article helpful?