|
Using data from a clinical trial of a commercial CAD system for lung cancer detection we separately analyzed the location, if any, selected on each film by 15 radiologists as they interpreted chest radiographs, 160 of which did not contain cancers. On the cancer-free cases, the radiologists showed statistically significant difference in decisions while using the CAD (p-value 0.002). Average specificity without computer assistance was 78%, and with computer assistance 73%. In a clinical trial with CAD for lung cancer detection there are multiple machine false positives. On chest radiographs of older current or former smokers, there are many scars that can appear like cancer to the interpreting radiologists. We are reporting on the radiologists' false positives and on the effect of machine false positive detections on observer performance on cancer-free cases. The only difference between radiologists occurred when they changed their initial true negative decision to false positive (p-value less than 0.0001), average confidence level increased, on the scale from 0.0 to 100.0, from 16.9 (high confidence of non-cancer) to 53.5 (moderate confidence cancer was present). We are reporting on the consistency of misinterpretation by multiple radiologists when they interpret cancer-free radiographs of smokers in the absence of CAD prompts. When multiple radiologists selected the same false positive location, there was usually a definite abnormality that triggered this response. The CAD identifies areas that are of sufficient concern for cancer that the radiologists will switch from a correct decision of no cancer to mark a false positive, previously overlooked, but suspicious appearing cancer-free area; one that has often been marked by another radiologist without the use of the CAD prompt. This work has implications on what should be accepted as ground truth in ROC studies: One might ask, "What a false positive response means?" when the finding, clinically, looks like cancer-it just isn’t cancer, based on long-term follow-up or histology.
|
