Development of methods to evaluate probability of reviewer’s assessment bias in Blinded Independent Central Review (BICR) imaging studies

J. Michael O'Connor; Manish Sharma; Anitha Singareddy

doi:10.1117/12.2512603

4 March 2019 Development of methods to evaluate probability of reviewer’s assessment bias in Blinded Independent Central Review (BICR) imaging studies

J. Michael O'Connor, Manish Sharma, Anitha Singareddy

Author Affiliations +

Proceedings Volume 10952, Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment; 109520P (2019) https://doi.org/10.1117/12.2512603
Event: SPIE Medical Imaging, 2019, San Diego, California, United States

Abstract

Purpose: To develop novel monitoring methods in Blinded Independent Central Review (BICR) imaging trials in which two radiologist reviewers assess the same images. In this project we aimed to ‘flag’ any reviewer that might have an assessment bias compared to the assessments of other reviewers on a specific study. Methods: Retrospective data analysis using R programming scripts was used to evaluate discordant assessments between two reviewers. We use a binomial test to determine the probability that an estimated low adjudication agreement rate is statistically less than the expected rate for all reviewer discordant assessment pairs. Results: We determined that for five or more discordant cases we can calculate the probability that each individual reviewer might have a statistically significant probability of low adjudication agreement for each discordant pair of assessments. We then analyzed the assessment data for sixteen oncological BICR clinical trials. Conclusions: The basic methods described can ‘flag’ or ‘signal’ a potential assessment ‘bias’. Although we initially focused on studies following one published clinical trial criteria to evaluate solid tumor we have applied the methods to other oncological studies with different published criteria which also may require double radiological reviews.

Conference Presentation

Citation Download Citation

J. Michael O'Connor, Manish Sharma, and Anitha Singareddy "Development of methods to evaluate probability of reviewer’s assessment bias in Blinded Independent Central Review (BICR) imaging studies", Proc. SPIE 10952, Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment, 109520P (4 March 2019); https://doi.org/10.1117/12.2512603

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available