Mr. Daniel Najarian Profile

Daniel Najarian

SPIE Involvement:

Author

Publications (1)

Proceedings Article | 3 April 2024 Poster + Paper

Effect of semantic distribution shift on AI knowledge retention in a sequential training paradigm

Daniel Najarian, Alexis Burgon, Nicholas Petrick, Berkman Sahiner, Kenny Cha, Ravi Samala

Proceedings Volume 12927, 1292729 (2024) https://doi.org/10.1117/12.3008265

KEYWORDS: Evolutionary algorithms, Semantics, Tumors, Data modeling, Image segmentation, Artificial intelligence, Algorithm development, Machine learning, Computer simulations

Read Abstract +

Many artificial intelligence algorithms are currently deployed in medicine to help decision making and treatment planning. As new data become available, these algorithms can be updated with the goal to increase performance and improve patient outcomes. One method for updating algorithms is incrementally training on newly available data in multiple steps following sequential training protocols. For segmentation algorithms, changing the class labels of background pixels is known as semantic distribution shift, which can decrease algorithm knowledge retention. In this work, we explore the effects of semantic distribution shift on the knowledge retention of sequentially trained tumor segmentation algorithms by systematically altering the reference standards to simulate annotations by different annotators. Algorithms were trained in two sequential steps. The first step used an unmodified reference standard and the second step training data had a modified reference standard. The modified reference standards, which simulate annotator over- and under-segmentation, were created by systematically dilating or eroding the reference standard with different kernel sizes. A baseline algorithm is trained with unmodified data in both steps. Two types of variability are explored. The first is homogeneous variability, where the modified reference standards are changed in a consistent manner; for example, all reference standard segmentations were dilated or eroded with the same dilation or erosion structural element. The second is heterogeneous variability, in which the modified training data contains a mixture of dilated and eroded reference standards. Algorithm performance is evaluated using the Dice Coefficient, 95% Hausdorff Distance, and Volume Distance Coefficient, an adaptation of the Volume Similarity Coefficient. For the gadolinium-enhancing tumor subregion, algorithms trained with homogeneous variability had Dice values ranging from 0.38±0.01 to 0.76±0.02, while algorithms trained with heterogeneous variability had Dice values ranging from 0.76±0.02 to 0.79±0.01, much closer to the baseline algorithm’s value of 0.81±0.03. The results show that semantic distribution shift due to homogeneous variability decreases algorithm knowledge retention more than the shift due to heterogeneous variability. In practice, this suggests that the potential decrease in knowledge retention resulting from the addition of new data can be minimized by ensuring that the new data comes from a diverse variety of sources.

View contact details

UPDATE YOUR PROFILE

Is this your profile? Update it now.

Sign into your SPIE.org account

Don’t have a profile and want one?

Create an account on SPIE.org

Keywords/Phrases

Search In:

Publication Years