So, with a 1-percent infection rate in the test population, a false-positive rate of only 0.5 percent leads to nearly 40 percent of the positive results being wrong. And although the false-negative rate is 50 times higher than the false-positive rate, it is nevertheless much more likely (nearly 160 times more likely) that a positive result will be wrong than a negative result will be wrong. Notice that this doesn't align with what most health authorities have been telling us, which is that we can trust a positive PCR result as proof that we're definitely infected, but that we can't rely on a negative result as proof that we're not infected. In fact, just the opposite is true – negative results are reliable and positive results are not – when the infection rate is low.

So, how often is the infection rate low enough for false positives to be a problem? The next slide shows the test positivity (what fraction of people tested received a positive result) in New York State from late March to September, and the percentage of positive results that would be wrong, assuming the same false-negative and false-positive rates of 25 and 0.5 percent. As you can see, when the disease was spiking, at the beginning of this period, there would have been very few false positives, and positive results could be trusted. However, as the infection rate and test positivity fell, in May, an increasingly large portion of the positive results would be wrong. Now that new cases in the U.S. are exceeding 100,000 daily, an unprecedented figure, the pendulum appears to be swinging the other way.


However, this data, doesn’t tell the whole story. Individuals who exhibit COVID-19 symptoms are more likely to be infected than individuals who have no symptoms. Thus, people who are tested because they have symptoms will likely have a higher test positivity, especially during an outbreak; for these people, positive results will be more reliable, more likely to be true positives, than the average data would indicate.

On the other hand, people who do not have symptoms will be less likely to be infected and will have a lower test positivity. For these people, positive results will be more likely to be wrong than the averages indicate. Thus, even during a major outbreak, there may be portions of the test population consisting of individuals that are mostly or entirely asymptomatic – individuals tested in nursing homes, homeless shelters, prisons, or other congregate living situations; patients tested automatically upon hospital admission or prior to surgeries; athletes tested as a requirement for participation in sports activities; etc. – for whom positive PCR results may likely be false.

Diagnostic applications are usually applied for chronic viral infections such as HCV, HIV and chronic HBV where symptoms or high-risk behavior initiates testing, although there are now screening recommendations for HCV. Still, in all these chronic diseases antibody concentrations are high and serology usually precedes rRT-PCR, so that false positives are rare. At present prevalence, COVID-19 testing is primarily widespread screening without confirmation.

For SARS-CoV-2 rRT-PCR, cycle threshold (Ct) of 24 or less has been shown to be highly predictable for identifying active COVID-19 cases (4), but since LoD of various methods drastically differ it is unclear which methods this applies to. Generally, methods do not amplify more than 40 cycles, but some systems go beyond 40 Ct. It seems likely that short probes in such systems could lead to amplification errors. Although there is no wide spread EQA proficiency programs for SARS-CoV-2, there is one report (5), of EQA in clinical laboratories for other RNA virus. The authors compiled 43 EQAs of rRT-PCR assays, conducted between 2004-2019. Each EQA involved between three and 174 laboratories, which together provided results for 4,113 blind panels containing 10,538 negative samples. 336 of the 10,538 negative samples (3.2%) were reported as positive. The authors defined the lowest percentage of the interquartile range which was 0.8% as a conservative estimate of the false positive rate. In another report, Sin Hang Lee found that 3 of 10 positive proficiency samples in the State of Connecticut were negative containing no SARS-CoV-2 RNA by a confirmatory assay (1). The Foundation for Innovative New Diagnostics (FIND) examined 22 rRT-SARS-CoV-2 diagnostic tests (6) and found diagnostic specificities ranging between 100% and 96% for 100 specimens assayed by each test. Although the great majority showed 100% specificity, given the small number assayed, the lower 95% confidence limit which was 95% for almost all assays would seem to be a better estimate (possible 5% error). Moreover, these were tested under controlled conditions, not at all similar to high output clinical laboratories running thousands of tests.

The Reverend Thomas Bayes (1701-1761) recognized a kind of statistic that predicts the posterior probability from the prior probability. For testing, this means the post test probability can be derived from the pretest probability if the prevalence is known. This sounds complicated but actually, Bayesian statistics are simple compared to classical frequentist statistics since one does not have to apply a null hypothesis, nor interpret p-values or effect-size and the results are obtained from simple mathematics. If, as discussed above (5), a 0.8% false positive rate is correct, at a six percent positive rate that some States claim, then there would be: 100 x 0.06 = 6 positives/100 tests. But if 0.8% are false positives, then only 5.2% are true positives with a positive predictive value (True positives/total positives x 100) of 5.2/6 x 100 = 86.6%. This means about 13.4% are false positive. Notice as the prevalence of disease decreases, the percentage of false positives to total positives increases because the true positive percentage decreases but the percent false positive (in this case 0.8%) stays the same. Thus, the percentage of false positives would be about 26.6% at a three percent positive rate.

The source of the problem is recognized from Bayesian analysis. If the prevalence is low (say a prevalence of 1%) even a very good screening test with 99% diagnostic specificity and 100% sensitivity will produce only 1% false positive results: (diagnostic specificity 1%) = 0.01 x 10,000 tests = 100 false positives/10,000 tests and (0.01% prevalence of disease at 100% sensitivity) = 0.01 x 10,000 = 100 true positive but for a poor positive predictive value of only 50% (100/200 x 100 = 50%). Recognizing this problem, the CDC suggests most testing should be diagnostic: “Considerations for who should get tested: People who have symptoms of COVID-19, people who have had close contact with someone with confirmed COVID-19, people who have been asked or referred to get testing by their healthcare provider, or state health department. Not everyone needs to be tested. (7)”

Because of rightful concern regarding disease transmission from asymptomatic and pre-symptomatic cases, this advice is not being followed. As a result, the great abundance of testing is screening not diagnostic. One way to reduce false positive results is to repeat the test using a test with a different format (different manufacturer). Due to limited testing facilities confirmation is not routinely performed and only a few positives are confirmed by a second rRT-PCR assay. I conclude it is likely that at current active disease prevalence the positive rRT-PCR results of many “asymptomatic” persons are false positives.

There are negative psychological implications of thinking one is infected when one is not and some persons with illness other than COVID-19 who test false positive might be hospitalized with COVID-19 patients and become infected. This may explain why some persons seem to have been infected twice: the first time being a false positive. It seems to me it is important for practicing medical professionals to be aware of these issues so that they can appropriately advise and direct suspect patients for additional testing.

REFERENCES

1. Lee SH. Testing for SARS-CoV-2 in cellular components by routine nested RT-PCR followed by DNA sequencing International Journal of Geriatrics and Rehabilitation 2020;2:69-96.
2. FDA. SARS-CoV-2 Reference Panel Comparative Data. Accessed 9/20/2020 doi: 10.1093/cid/ciaa638 https://www.fda.gov/medical-devices/...mparative-data.
3. Maske M. NFL’s 77 positive virus tests were ‘likely false positive results,’ company says. Accessed 9/20/2020 https://www.washingtonpost.com/sport...us-tests-new-j
4. Bullard J, Dust K, Funk D, Strong JE, Alexander D, Garnett L, et al. Predicting infectious SARS-CoV-2 from diagnostic samples. Clin Infect Dis 2020 doi: 10.1093/cid/ciaa638.
5. Cohen AN, Kessel, B. False positives in reverse transcription PCR testing for SARS-CoV-2. Accessed 9/20/2020 https://doi.org/10.1101/2020.04.26.20080911 (not peer reviewed).
6. Accessed 10/01/2020 https://www.finddx.org/covid-19/sarscov2-eval-molecular 7. Accessed 10/01/2020 https://www.cdc.gov/coronavirus/2019- ncov/testing/diagnostic-testing.html

Scientific Shorts are brought to you by the