The combination of AI and radiologists found to increase radiologists’ screening accuracy for breast cancer detection
The combination of an artificial intelligence (AI) system and a radiologist provides better screening accuracy for breast cancer, as evidenced by higher sensitivity and specificity according to the findings of a retrospective analysis by an international team of radiologists.
The use of screening mammography is intended to identify breast cancer at an earlier stage as the treatment will be more successful. Moreover, in recent years there has been an increasing interest in the use of AI systems and a recent study found that using an AI system outperformed all human readers, with a larger area under the receiver operating characteristic curve of 11.5% for screening mammograms of breast cancer. Nevertheless, a 2021 systematic review considering the use of AI for image analysis in breast cancer screening programs concluded that the current evidence for AI does not yet allow assessing its accuracy in breast cancer screening programs, and it is unclear where on the clinical path AI could benefit most† Other work considering the role of AI for breast cancer screening suggested that: an AI system can correctly identify a portion of a screening population as cancer-free and also reduce false positives and therefore has the potential to improve the efficiency of mammography screening†
But what if an AI and radiologists worked together so that the AI could initially review scans and identify normal cases, but those suspected of cancer and where diagnostic uncertainty existed were referred to the radiologist? This was the question addressed in the research team’s retrospective analysis. The system was designed so that the AI system would flag possible cancer scans, and where it was unsure of the diagnosis, for a second reading by a radiologist. The team first trained the AI system using an internal dataset and then used an external dataset and compared its interpretation with that of a radiologist. The performance of both the AI and the radiologists was assessed for sensitivity and specificity, and the test sets contained a mix of both normal and cancer scans.
AI and radiologists combined performance
For the external data set, the radiologist had higher sensitivity (87.2% vs 84.6%, radiologist vs AI system) and specificity (93.4% vs 91.3%) and in both cases this difference was statistically significant ( p < 0.001 for both).
However, when the AI and radiologists worked together, the radiologist’s sensitivity was 89.7% and specificity was 93.8%. In other words, the combination improved both sensitivity and specificity. The authors calculated that this corresponded to a triage performance, ie the fraction of scans that could be automated) of 60.7%.
Based on these findings, the authors concluded that their system leverages the power of both the radiologist and the AI system and has the potential to improve radiologists’ screening accuracy.
Leibig C et al. Combining the powers of radiologists and AI for breast cancer screening: a retrospective analysis Lancet Figure Health 2022