Effective Techniques to Eliminate False Positives in Data Analysis
Athletes will often claim that a test revealing that they have been using performance-enhancing drugs such as steroids is a false positive. A memorable example is Floyd Landis, the American cyclist who won the 2006 Tour de France and was later stripped of his title.
Landis had done well in the race, but then performed poorly near the end, falling back to 11th place. Despite severe pain from an illness that left him with a crumbling hip and bone rubbing on bone, he finally stormed back to win the tour by 57 seconds. When later tests indicated that Landis had used a banned substance, he claimed the test was a false positive caused by the medication he was using for his hip problem. The controversy went on for months, fuelled by uncertainty over the
reliability of the medical testing technology.
Go deeper with GlobalData
Such issues may seem remote from medical practice, but they are actually highly relevant. A medical test yields a false positive when it indicates abnormal results for a healthy patient. That is, a patient gets an abnormal test result, but no illness is later detected. In clinical settings, false positives frequently occur in tests for prostate cancer, pregnancy and drug use. False positive pregnancy tests are common enough to be a regular plot device in popular television programmes.
In the context of scientific research, perhaps the best-studied false positives are those that occur in screening mammography. This test is routinely done on healthy adult women (usually from the age of 50) to detect breast cancer in its early stages, so that it can be treated. Among younger women, who have low rates of breast cancer, as many as nine in ten abnormal mammograms are false positives.
It has been estimated that after ten years of regular mammograms, US women have a 50% chance of
getting a false positive.
Mammogram quality varies substantially from region to region. In the Netherlands, 1–2% of mammograms are abnormal, but US rates are ten times higher. A woman’s chances of getting a false positive mammogram can be even higher than that, because false positive rates are higher for some facilities and for some women. The higher rate of abnormal test results doesn’t help American women live longer: they die just as often from breast cancer as Dutch women.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataPOSITIVE HARM
But if false positives aren’t helping, are they doing any harm? The evidence clearly shows that they are. A recent meta-analysis published in the Annals of Internal Medicine found that false positive mammograms cause great distress to women. Some women report anxiety about getting breast cancer several years after receiving false positive mammograms.
Some caution is warranted, here, because these were not clinically severe cases of psychological illness; they were very specific to thoughts about breast cancer. That said, a dramatic report in the BMJ documented the suicides of two women who had received false positive mammograms. Other research has reported negative effects from false positive screening tests for prostate, lung and cervical cancer. False positives cause inconvenience and possibly harm to the women and men who must undergo
additional testing, procedures and surgery. Follow-up procedures can be relatively benign, such as a sonogram or another mammogram, but they can also include invasive procedures, such as a needle biopsy of the breast or a procedure that requires cutting into the breast.
The discussion thus far has focused on the harm done by test results that turn out to be erroneous. However, correct tests can also cause harm. Research suggests that many of the health problems detected in medical tests will not lead to cancer or other serious illness, which means that some of the mastectomies and other medical treatments carried out after positive tests are unnecessary.
False positives are expensive. Some estimates suggest that false positive mammograms alone cost over $100m annually in the US. Some of this cost is passed directly on to patients through co-pays, deductibles and other costs not covered by insurance. Also, whatever costs are covered inevitably force the cost of insurance to rise. In countries where healthcare is paid for by the state, taxes are increased.
THE TROUBLE WITH TESTING
The reasons for so many false positives are well understood. One reason has to do with base rates. When testing for any rare outcome, many more healthy adults than sick ones will be tested. Inevitably, no matter how good the test (and many are far from great), it will yield many false positives. Few women have breast cancer (less than 1%), and the test is good at detecting the disease when it is present in this small group of women. But the test also wrongly detects the disease in the much
larger group of women who do not have the disease, resulting in a high number of false positives relative to correct positives.
A related issue is that patients are becoming consumers, demanding more medical tests. Health spas offer weekend specials that include massage, special meals and full body MRIs. These new recreational tests are certain to indicate problems that do not bear up under further medical scrutiny.
A second factor is that more accuracy in the detection of a disease almost always comes at the cost of less specificity when it comes to saying the disease is absent. As medical tests become more numerous and tests are used more often, this lack of specificity in terms of confirming the absence of disease will lead to more false positive test results.
For example, the new MRI technology that shows great promise for detecting breast tumours is less specific when it comes to establishing the absence of tumours and will thus produce more false positives. Similarly, Pap smear tests are being supplemented with, and may one day be replaced by, new DNA typing tests for HPV, the virus that causes cervical cancer. These new tests produce more false positives.
Another reason for the increase in false positives is cost saving. In countries such as the US, often a single technician will read mammograms because this is cheaper for the medical practice than having two technicians read them, as is common in countries such as the Netherlands, where the state pays for mammography screening.
Litigation is also driving the increase in false positives. In the US, missed cancers are the main cause of medical malpractice lawsuits. Medical personnel are understandably worried about being sued for missing signs of cancer, and their safety-conscious mammography readings drive up the rate of false positives, even though they don’t help people live longer. As litigation and insurance costs soar, we can expect test interpretations to become more conservative, boosting the rates of false
positives.
POSITIVE ACTION
False positive test results can do considerable harm to patients. So what can be done? Some of the most potent solutions for reducing the number of false positive tests can only be planned and implemented by governments or perhaps large healthcare organisations. However, there are several evidence-based ways for individual hospitals and medical practices to reduce the harm caused by false positive medical tests:
Better reading. One obvious step is to improve the way tests are performed. This can be done by using well-adjusted equipment, and experienced and well-trained technicians. Feedback reports and incentives for quality can also improve reading. These changes can be implemented by individual medical practices and hospitals, but some are especially effective when implemented regionally or nationally.
Use two readers. Having two technicians independently interpret a test output can, for tests such as mammograms, increase accuracy. The increased cost for the mammogram provider is, of course, the problem with this solution.
Use previous records. False positive mammogram test rates can be halved if new mammograms are compared with older ones from a previous test. Patients can be educated about the value of bringing older films when visiting a new physician for the first time.
Raise the risk level. Testing people at low risk of disease leads to a high rate of false positives. For example, in the US, screening mammography begins for women at the age of 40, but most other countries start at 50. Screening so many 40–49-year-olds, who have the lowest incidence of breast cancer, ensures high false positive rates. Raising the screening starting age to 50 (that is screening higher-risk women) is a simple and safe way to reduce false positives.
Provide faster follow-ups. Studies show that women suffer less anxiety if their abnormal mammograms are cleared on the same day as their test. Many centres providing testing services are now adopting such a procedure.
False positive results are a risk in any medical test, even the best ones. Decisions about what is an acceptable rate of false positives will inevitably be influenced by the usual cost concerns, but they should also take into account the negative physical and psychological effects of false positives on patients. Simple strategies can reduce such harm to patients while maintaining high quality.
