Technology can't replace doctors' judgment in reading mammograms
Radiologists should not become too dependent on the use of computer-assisted detection (CAD) technology when reading screening mammograms because the doctors can see lesions that CAD sometimes misses. This is according to a study conducted at Group Health Cooperative, a Seattle based health care system. The research appears in the December issue of the American Journal of Roentgenology.
"Our study shows that radiologists must continue to rely on their own judgment when determining whether lesions seen on mammograms require further testing," said Stephen Taplin, MD, MPH, who led the research at Group Health before joining the National Cancer Institute as a senior scientist.
CAD uses computer software to identify and mark areas of concern on mammograms. Radiologists typically review the CAD-marked images after they interpret the original film.
While early CAD evaluations showed it improved cancer detection, more recent studies have raised questions about CAD's performance. For example, while it is believed that CAD alerts radiologists to potential areas of concern, experts have wondered whether CAD too frequently marks normal areas rather than only identifying problem areas that the radiologist should have detected.
To answer these questions, the researchers at Group Health designed a study using a sample from more than 56,000 screening mammograms taken between 1996 and 1998. By identifying cases of breast cancer diagnosed within two years after the mammograms were taken, they created a total set of 441 mammograms from three different groups. Included were mammograms from women who:
1) remained cancer-free two years after their mammograms
2) developed breast cancer within one year, or
3) developed breast cancer within 13 months to two years.
The sample was then used to test the performance of 19 radiologists, each of whom read 341 mammograms with and without CAD. The researchers then compared the results of the two approaches for each mammogram.
This is the first study of CAD using a random sample of cases from a screened population rather than using selected cases of visible cancers. In this way, it more closely resembles the way that radiologists use CAD in real practice.
The study showed that CAD assistance increased radiologists' ability to determine that a woman without cancer was, in fact, cancer free—a quality known as mammographic "specificity." Overall specificity increased from 72 percent without CAD to 75 percent with CAD. This 3 percent difference means that CAD allows 30 women in every thousand women screened to avoid further evaluation.
CAD assistance did not affect the radiologists' overall ability to spot cancer where it was present—a quality known as mammographic "sensitivity." The doctors performed equally well with and without CAD.
However, CAD does not mark all visible abnormalities. And when the researchers analyzed the radiologists' performance on mammograms with lesions that CAD did not catch, they found that the doctors were less likely to recommend further evaluation when they were using CAD than when they were not using CAD.
"This means that the radiologists may have been deferring to CAD and believing its interpretation rather than their own interpretation," said Taplin. "This is something the originators of the technology say radiologists should not do. This study shows that it is hard to ignore the technology, and it raises the question of whether there is a potential for CAD to do harm."
Taplin and his co-authors recommend training for radiologists that focuses on characteristics that CAD may miss, namely "masses, asymmetries, and architectural distortions" visible on the mammograms. They also note that research into these visible, unmarked lesions may offer the best chance to improve CAD-assisted mammography.
The researchers also found that breast density—a measure of the amount of fat tissue in the breast—did not affect CAD's performance.
Source: Group Health Cooperative Center for Health Studies
