Evaluating Performance

The evaluation of any new diagnostic test involves comparison to an appropriate "gold standard" test and determination of the percentage sensitivity, specificity, and positive and negative predictive values. Prior to determining the performance characteristics on clinical specimens, the analytical sensitivity of a test, that is, the smallest amount of analyte that can be detected by the test, should be determined. For PCR, this is usually expressed as pg or fg purified DNA and is easily determined by testing serial 10-fold dilutions of target DNA. The most sensitive PCRs can detect 0.1 fg target or the equivalent of 1-5 target copies (Erlich et al., 1991; Mahony et al., 1993a). The sensitivity of M-PCR for each target can be determined by preparing a five-member sensitivity panel consisting of serial 10-fold dilutions of each target DNA pooled together, ranging from 1 pg to 0.1 fg (Fig. 1). Once the analytical sensitivity has been determined, the sensitivity and specificity can be determined using clinical specimens. A sample Chlamydia-Neisseria M-PCR result obtained with genitourinary specimens is shown in Fig. 2. An approximation of the sensitivity can be determined quickly by testing 10-20

M 1 23456789 10 tl M

M 1 23456789 10 tl M

Figure 2 Detection of Chlamydia trachomatis and Neisseria gonorrhoeae DNA in genitourinary specimens by M-PCR. Outer lanes contain 1-kb marker DNA. Locations of 241-bp and 390-bp amplicons of C. trachomatis and N. gonorrhoeae, respectively, are indicated by arrows. Specimens in Lanes 3, 6, and 9 are positive for C. trachomatis. Specimens in Lanes 1, 4, 5, and 8 are positive for N. gonorrhoeae. Specimens in Lanes 2, 7,10, and 11 are positive for both C. trachomatis and N. gonorrhoeae.

Figure 2 Detection of Chlamydia trachomatis and Neisseria gonorrhoeae DNA in genitourinary specimens by M-PCR. Outer lanes contain 1-kb marker DNA. Locations of 241-bp and 390-bp amplicons of C. trachomatis and N. gonorrhoeae, respectively, are indicated by arrows. Specimens in Lanes 3, 6, and 9 are positive for C. trachomatis. Specimens in Lanes 1, 4, 5, and 8 are positive for N. gonorrhoeae. Specimens in Lanes 2, 7,10, and 11 are positive for both C. trachomatis and N. gonorrhoeae.

specimens that are positive by the gold standard test. This result can be used to determine the sensitivity of the test and, more importantly, whether any substance in the clinical specimen is interfering with the PCR. The sensitivity and specificity of the PCR can then be determined, first in a retrospective fashion using stored specimens, then prospectively using specimens collected sequentially from a clinically defined cohort of patients. Further evaluations of the test in various populations with different prevalences of infection (low, medium, and high) and different clinical presentations should be performed to assess the effect of disease prevalence (Sackett et al., 1985) and spectrum bias (Lachs et al., 1992) on test performance. For M-PCR tests, a sufficient number of specimens should be tested so a minimum of 15-20 positives is obtained for each virus or bacteria. If the numbers are small, 95% confidence intervals should be used for sensitivity, although this is rarely done.

Even the best designed PCR assays occasionally develop problems. Many problems are the results of changing reagents without properly testing them, as described in Section IV. Problems due to carryover contamination can be devastating and set a laboratory back 1-3 mo depending on the number of primers in use and the laboratory's knowledge of the problem. (See Section IV,A for ways to prevent carryover contamination.) Sometimes a particular

E. Troubleshooting

PCR assay works well with one set of primers, then suddenly gives many nonspecific products with a different set of primers or a new batch of the same pair of primers. The introduction of a new lot of previously used primers should follow an appropriate testing protocol to verify its performance and determine whether it is having any negative influence on sensitivity (see Section IV,B). Evaluation of a novel primer pair may reveal many nonspecific products that prevent visualization of the expected amplicon on ethidium bromide-stained gels and, more importantly, result in a marked decrease in analytical sensitivity because of incorporation of primers and nucleotides into nonspecific products. Nonspecific products can be eliminated in part or completely by raising the annealing temperature, which increases the fidelity of primer-target annealing and prevents primer dimer formation. If raising the annealing temperature does not eliminate nonspecific amplification products, raising or lowering the MgCl2 concentration or decreasing the primer concentration, as described in Section III,B, may alleviate the problem. If none of these maneuvers improves the specificity of amplification, there may be too much target DNA in the sample, reducing the DNA concentration by making serial dilutions of the sample sometimes improves specificity. Using "hotstart" or uracil-AT-glycosylase methods alone or together has been shown to increase the specificity of amplification (Erlich et al., 1991; Chou et al., 1992; Thornton et al., 1992). The hot-start involves raising the reaction temperature to a temperature above the Tm of the primers (usually 80°C), then adding the Taq polymerase or primers to the reaction tube (before cycling begins) to prevent the primers from annealing to noncomplementary DNA, thereby eliminating nonspecific products. Although useful in some PCRs, the hotstart method does not circumvent problems attributable to poorly designed primers that may contain short stretches of bases that appear in nontarget DNA present in the specimens (i.e., host derived). The use of paraffin wax to separate primers and enzymes physically from DNA in reaction tubes uses the same principle of separating primers and DNA to improve the fidelity of primer annealing.

Inhibitors of Taq polymerase may be present in clinical specimens, preventing the detection of specific viral nucleic acids by PCR. Although these inhibitors have been poorly characterized, they are usually proteinaceous or nonproteinaceous substances that inhibit polymerases nonspecifically, presumably by altering the quaternary structure of the polypeptide or by interfering with the active site of the enzyme. Certain salts such as oxalates and urea, heparin, iron-containing compounds such as hemoglobin, and urine have been reported to inhibit Taq polymerase (Mercier et al., 1990; Holodniy et al., 1991; Khan et al., 1991). Hemoglobin can be particularly troublesome if plasma, serum, or blood is being tested by PCR. Traces of hemoglobin can be removed by deproteination with organic solvents such as phenol or chloroform. Inhibitors of PCR can be removed by selective adsorption of nucleic acids to hydroxyapatite, silica gel, or immobilized captive DNA.

We have had success testing some urine and semen specimens that contain inhibitory substances by boiling or by making serial dilutons, then retesting by PCR. For the latter method, the idea is to dilute out the inhibitor before diluting out the target sequence so amplification can be done. This simple approach has not, however, been rigorously analyzed to ensure that target sequences are also not lost to dilution. The possibility of selective inhibition of one primer pair over another also has not been investigated systematically (Coultee et ai, 1991). These concerns accentuate the need for appropriate quality assurance measures for both conventional PCR and M-PCR.

Was this article helpful?

0 0

Post a comment