Evaluate Available Diagnostic Tests For Causality (Apply Case Series Imputation Procedure)
Figure 18.1 Use of imputation methods to carry out two-phase signalling.
further action exists. Like other case series formation methods, case series imputation is intended to summarize the aggregate evidence in favor of a product-AE relationship that is exhibited by a group of cases, i.e. a group of reports that meet a case definition (see the section Spontaneous Signalling Methods Used at the Case Series Formation Step in Chapter 19). While the use of imputation methods for these two purposes was not clearly delineated in the past, it has recently been pointed out and discussed by Meyboom (Meyboom et al., 1997b; Meyboom, 1998).
Regardless of functional intent, imputation methods are dependent on report quality (accuracy and completeness (Edwards et al., 1990)), which differs from one data source to the next. Individual patient reports from clinical trials are usually well documented on standardized data forms and offer access to medical records. On the other hand, patient reports from studies can also be limited by investigators' perceptions of reportable content, the lack of data following closure of a trial, and the forwarding of a large number of reports for which a medical rationale may be lacking. Literature case reports are characterized by rigorous editorial expectations. However, the publication process does not impose any clearly definable minimum standard, is not applied uniformly, and, in general, not does not provide for peer review of actual medical records (Venulet, 1985). While spontaneous individual patient reports frequently lack sufficient detail to provide
Table 18.2. Two uses for imputation in adverse event surveillance schemes.
Case series imputation
Signalling step Purpose
Resource expenditure Depends on
Report case requirements
Single reports in a report database
Screen for ''interesting'' product-AE pairs
Semi- or fully-automated
Systematically entered data fields pertaining to tests for causality
Must meet minimum requirements to be a screenable report
Multiple cases in an assembled case series
Case series formation
Summarize evidence in favor of product-AE relationships
Time and resource intensive
Must meet a case definition to be included in an analysis value and are often documented with practitioner opinions only, prescriber-selected patient reports have repeatedly proven effective in finding important safety outcomes. Additionally, follow-up activities can compensate for initial deficiencies in spontaneous reports by seeking out and acquiring supporting medical documentation (Clark et al, 1990). Thus, each of the three major reporting environments that give rise to individual patient reports are associated with strengths and limitations, and each has come to occupy a complementary niche in health care monitoring systems.
The Role of Imputation Methods in AE Surveillance Programs
Defining a logical role for the use of imputation methodology in AE surveillance programs has been complicated by its historical development. Single report imputation methodology was first published in the medical literature in the 1970s, a time that coincided with the adoption of early computerized expert systems into medical practice. Most of this work pertained to single AE occurrences (as distinct from a series of the same AE in different patients), originated in academic settings, and was not designed to address the needs of surveillance systems (Irey, 1976a; Karch and Lasagna, 1977; Kramer et al., 1979; Naranjo et al., 1981). Influenced by these trends, AE-
oriented imputation instruments were developed primarily as single occurrence, medical diagnostic aids, and were only subsequently adapted for use in surveillance databases.
Since the main focus of AE surveillance is to find product-AE pairs for possible public health interventions, and not to establish medical diagnoses in individual patients, the published imputation literature is only indirectly applicable to AE surveillance systems. At present, in-depth, single report imputation procedures in AE monitoring are essentially limited to satisfying regulatory requirements (Begaud and Royer, 1986) or addressing special project needs. For imputation methods to become widely applicable as screening imputation tools, they would have to be simplified for automated or semi-automated use, and would have to be adapted to a report database environment (Venulet, 1992).
Likewise, the bulk of imputation-based methodology involves the assessment of product-AEs on a case-by-case basis, and is also minimally applicable to case series imputation procedures. While single cases of exceptional value can rarely become pertinent signalling evidence (Meyboom et al., 1997a), case series imputation is almost always based on multiple patient occurrences of the same AE (Meyboom et al., 1997b). Except in highly unusual circumstances, product-AE associations have generally not been proposed on the basis of one known patient experience. For imputation methods to become widely applicable as case series imputation procedures, they would have to be extended from one to multiple cases, and would have to produce results that are interpretable from a population viewpoint.
Imputation, Causality, and Causality Assessments
Since imputation addresses causally oriented data by way of defined procedures, it has become closely identified with the phrase "causality assessment". Unfortunately, the word "causality" is confusing because it describes at least six different, but loosely related, concepts: (1) etiolo-gic (causal) certainty, (2) epidemiologic causality, (3) retrodictive causality, (4) operational causality (causality assessment procedures), (5) regulatory causality, and (6) legal causality. Without clarification, the meaning of the word "causality" is too general to be of value when describing AE surveillance procedures or results.
Imputation screening procedures are specialized methods for rating operational causality. Their purpose is to screen surveillance databases for suspected product-AE pairs. In contrast, case series imputation methods are best grouped together with epidemiologic methods, since they are intended to clarify, however imperfectly, epidemiologic propositions. (As a result of constraints related to data quality and completeness, the term "pre-epidemiology" has sometimes been used in place of the word "epidemiology" to describe their use (Wartenberg and Greenberg, 1993).) Case series imputation is one of several case series formation signalling methods, all of which assess potential product-AE relationships through the analysis of incompletely reported case series (see Chapter 19).
Etiologic (Causal) Certainty
Etiologic certainty accepts without qualification the retrodictive causal proposition that an AE occurring in a particular patient would not have happened as and when it did unless a specified product exposure had occurred (Hutchinson and
Lane, 1989; Kramer and Lane, 1992). This is equivalent to the statement that a particular product caused a particular AE to occur in a specific patient. A causal versus a non-causal AE in this sense amounts to acceptance or rejection of an absolute etiologic proposition, and is reducible to either "yes" or "no". It is widely accepted, however, that causation for a given AE type is multifactorial and variable, so that, in a given individual, the extent and nature of the causal constellation is unknown (Rothman and Greenland, 1998). Although the word "causal" continues to be used in the sense of etiologic certainty, this concept is no longer scientifically viable as a causal model.
Over the course of the twentieth century, epide-miologic causality gradually displaced etiologic certainty as the prevailing causal model for medical applications. Epidemiologic causal models possess time-space context and counterfactuality (Greenland and Rothman, 1998). Time-space context implies that rates or proportions pertaining to safety outcomes can be ascertained and compared across product exposures because the temporal and geographical limits of the model are explicitly defined. This, in turn, means that epidemiologic causality refers to aggregate comparisons, and need not be ascribed to any particular case. Epidemio-logic causality also depends on the notion of counterfactuality, i.e. determining what would have happened to a particular group of cases under the scenarios of both an index and alternative exposure. Since both scenarios cannot occur simultaneously, the observer must have confidence that another aggregate case experience (such as a control group) can be substituted for one of the two counterfactual scenarios as a comparative standard. Taken together, time-space context and counterfactual reasoning allow causal propositions to be formally evaluated by comparing model-based test statistics vis-à-vis their expected values. Epidemiologic models allow safety professionals to quantify the strength of probabilistic evidence in favor of a product-AE relationship, rather than relying on personal judgment to formulate absolute responses on a case-by-case basis.
Individual cases of product-AEs can also be considered in a counterfactual paradigm that has been assessed in its most detailed way with a Bayesian causal/non-causal odds model (Lane et al., 1987; Kramer and Lane, 1992) (see Figure 18.1). The Bayesian odds model uses individual case information and epidemiologic data to assess individual cases. The resulting quantity, which is applicable to a particular case only, can theoretically demonstrate the strength of probabilistic evidence in favor of a product-AE causal argument for a single patient (i.e. it addresses a retrodictive causal proposition). Unfortunately, systematic formal statistical testing of retrodictive causal propositions is unrealistic in surveillance environments for two major reasons. First, the Bayesian odds model requires existing epidemio-logic data pertaining to product-AE causality. But in AE surveillance, such data usually do not exist, or, if they do, often represent only a "best guess'' estimate. Second, the sensitivities and specificities for available diagnostic testing for causality for a particular case can be difficult to quantify. Problems with data availability may account for why authors analyzing specific retro-dictive causal hypotheses have not recommended formal statistical testing procedures for individual product-AE cases vis-à-vis an expected value, and have, instead, treated the results as if the underlying variability of the estimate was not calculable (Lane et al., 1987; Ghajar et al., 1989). Given these difficulties, from the surveillance viewpoint the Bayesian odds model and its derivatives are best considered to be forms of operational causality (see below), and not as models that offer precise solutions to retrodictive causal propositions (Hutchinson and Lane, 1989).
Causality assessment procedures are a group of methods by which single cases can be considered in a counterfactual model for the purpose of generating operational causal ratings (Venulet, 1992). All operational causality instruments are based on the Bayesian principle that individual patient data can modify an assessor's pre-existing view of a causal association between a particular etiologic alternative and an AE to obtain a more accurate final impression of a particular patient's adverse experience. The operational rating that results from this process can then be used to categorize individual patient product-AEs along an arbitrary continuous or ordinal scale. Although such ratings provide causal information within the scale defined by the observer, they are not referable to a time-space context, and cannot mathematically relate individual patient causality to a larger causal concept by way of formal hypothesis testing. From the surveillance perspective, operational causality ratings are useful as triaging methodologies (i.e. imputation screening tools) that help to focus subsequent analytic activity on high priority patient experiences.
Regulatory causality means the use of an operational causality instrument to explicitly define one or more categories as "causal" for reporting purposes. Since regulatory causality depends on how the results of an operational instrument are further defined as a basis for reporting actions, it is not necessarily equivalent to the results of any specific operational causality assessment. Regulatory causality implies that a reporting entity has defined a procedure in order to comply with territorial requirements.
In the context of product-AEs, legal causality means the designation of an AE as causal during proceedings conducted by a court possessing appropriate jurisdiction (Freilich, 1984). This legal view of causality may or may not coincide with any other view of causality that is formulated on either an aggregate or individual case basis. The usual purpose of a jurisprudential causal debate pertaining to a health care product is to adjudicate claims against manufacturers that have been filed by an individual or group under product liability law.
Was this article helpful?