Accurate classification of medical images is crucial for disease diagnosis and treatment planning. Deep learning (DL) methods have gained increasing attention in this domain. However, DL-based classification methods encounter challenges due to the unique characteristics of medical image datasets, including limited amounts of labeled images and large image variations. Self-supervised learning (SSL) has emerged as a solution that learns informative representations from unlabeled data to alleviate the scarcity of labeled images and improve model performance. A recently proposed generative SSL method, masked autoencoder (MAE), has shown excellent capability in feature representation learning. The MAE model trained on unlabeled data can be easily tuned to improve the performance of various downstream classification models. In this paper, we performed a preliminary study to integrate MAE with the self-attention mechanism for tumor classification on breast ultrasound (BUS) data. Considering the speckle noise, image quality variations of BUS images, and varying tumor shapes and sizes, two revisions were adopted in using MAE for tumor classification. First, MAE’s patch size and masking ratio were adjusted to avoid missing information embedded in small lesions on BUS images. Second, attention maps were extracted to improve the interpretability of the model’s decision-making process. Experiments demonstrated the effectiveness and potential of the MAE-based classification model on small labeled datasets.
Ultrasound imaging is an effective screening tool for early diagnosis of breast tumor to decrease the mortality rate. However, differentiation of tumor type based on ultrasound images remains challenging in the field of medical imaging due to the inherent noise and speckles. Thus, obtaining additional information for lesion localization could better support the decision-making by clinicians and improve diagnosis fidelity. Recently, multi-task learning (MTL) methods have been proposed for joint tumor classification and localization, where promising results were demonstrated. However, most MTL methods trained independent network branches for the two different tasks, which might cause conflicts in optimizing features due to their different purposes. In addition, these methods usually require fully-segmented datasets for model training, which poses a heavy burden in data annotation. To overcome these limitations, we propose a novel MTL framework for joint breast tumor classification and localization, motivated by the idea of attention mechanism and weakly-supervised learning strategy. Our method has three major advantages. First, an auxiliary lesion-aware network (LA-Net) with multiple attention modules for lesion localization was designed on top of a pre-defined classification network. In this way, the extracted features for classification were directly augmented by the region of interest (ROI) predicted by the LA-Net, alleviating the potential conflicts between the two tasks. Second, a sequential training strategy with a weakly-supervised learning scheme was employed to train the LA-Net and the classification network iteratively, which allows the model to be trained on the partially-segmented datasets and reduces the burden on data annotation. Third, the LA-Net and classification network design are modularized so that both architectures can be flexibly adjusted for various applications. Results from experiments performed on two breast ultrasound image datasets demonstrated the effectiveness of the proposed method.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.