The WHO has defined a signal as: "Reported information on a possible causal relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously." An additional note says: "Usually more than one report is required to generate a signal, depending on the seriousness of the event and the quality of the information'' (Edwards and Biriell, 1994).
A signal is therefore very tentative in nature; the first expression that something might be wrong with a medicinal product, or a hint given by new information that might support or explain a medicinal product/adverse reaction relationship already known.
Both quantitative and qualitative factors come into the decision of whether something is a signal or not (Edwards et al., 1990). Many algorithms have been proposed for determining causality between a signal and an adverse reaction, but there is no perfect way of doing this that fits all possible situations. Perhaps the use of the Bayes-ian approach proposed and developed by Auriche (1985) and Naranjo and Lancetot (1991) is the most attractive, since Bayesian logic allows one to build up a pattern of probability that changes according to the addition of new information. This intuitively fits the clinical diagnostic approach, and is transparent.
Apparent causality in a single case, or even a series, is not the only issue in comprehensive early signal detection. One might exclude many of the case reports with limited information, yet, because a case record does not allow for remote assessment of the case, this does not mean that the original observer was incorrect, only that one cannot confirm the observation. Thus, the quantity as well as the quality of reports of associations is valuable.
The use of ''poor quality" reports as a trigger for a signal should be considered more carefully if the clinical event is serious. Early warning is more important, and a signal based on doubtful evidence should promote the search for better.
There may be certain items of information within a set of reports that trigger consideration of a signal other than just the medicinal product and clinical event. It might be the apparent over-representation of higher doses of the relevant drug, concomitant treatment, or certain patient characteristics.
The above are just some of the common reasons for someone to consider during the evaluation of an early signal. There are many others, such as the finding of a problem with one medicinal product that triggers a search into products with similar effects. What is clear is that there are very complex interacting patterns of information that may trigger ideas.
Apart from the complexity of possible important patterns in data, the volume of case reports on suspected medicinal product adverse reactions is massive. The WHO Programme for International Drug Monitoring database holds nearly 3 million case reports. There is more in the published literature and even more from varieties of clinical studies. One begins to see the problem as looking for the proverbial "needle in a haystack'' (Edwards, 1997).
If the above does not make the problems daunting enough, we must see medicinal product safety in the context of the use of those products. We need to know not only the numbers of people exposed to the products, but also why they were used, in what kind of patients, for what reason, and with what outcome.
The human brain is excellent at finding significant patterns in data: humans would not have survived if that were not so! On the other hand, the vast quantities referred to above cannot be usefully observed, let alone held in the memory for a person to analyse. Many people are involved in pharmacovigilance, but we are not yet wise enough to divide up the great task we have. Even if we did, there would still be a place for bringing the data we have to us for analysis in ways that allow us to see patterns more easily, and without our preconceptions blinding us to see things only in a certain way, conditioned by our experience.
It is true that in looking for significant patterns by sifting through data, one will eventually come up with something that looks significantly probable by chance: data "dredging" or "trawling" or a "fishing expedition" is bound to catch something, but not much that is useful. In trying to find signals, this view is too rigid; first, since one acknowledges that an early signal is tentative, that simply urges further work to be performed on that hypothesis. Secondly, from experience, a principal argument has evolved in drug safety, namely if important signals are not to be missed, the first analysis of information should be untrammelled by prejudice and rigid protocols. Thirdly, and notwithstanding the second point, data mining is not necessarily a random rummaging through data in an aimless fashion, which is what the term "dredging" implies. It is certainly true that the involvement of objects and the characterisation of any relationships in advanced pattern recognition is largely unsupervised, but the level of supervision and the kind of logic that is applied to data is flexible and transparent: this can be compared with the conventional use of "mining", which is defined as "a system of excavations made for the extraction of minerals''. In essence we consider that data dredging should be used as a pejorative term for unstructured fiddling about with data, or worse, the application of a structure to data to make it fit a biased hypothesis in a way to give added credibility to the result. Data mining, on the other hand, should be considered as a term for the application of a tool or tools to analyse large amounts of data in a transparent and unbiased fashion, with the aim of highlighting information worth closer consideration.
Was this article helpful?