Recent Foundation Models have begun to yield remarkable successes across various downstream medical imaging applications. Yet, their potential within the context of multi-view medical image analysis remains largely unexplored. This research aims to investigate the feasibility of leveraging foundation models for predicting breast cancer from multiview mammograms through parameter-efficient transfer learning (PETL). PETL was implemented by inserting lightweight adapter modules into existing pre-trained transformer models. During model training, the parameters of the adapters were updated while the pre-trained weights of the foundation model remained fixed. To assess the model's performance, we retrospectively assembled a dataset of 949 patients, with 470 malignant cases and 479 normal or benign cases. Each patient has four mammograms obtained from two views (CC/MLO) of both the right and left breasts. The large foundation model with 328 million (M) parameters, finetuned with adapters comprising only 3.2M tunable parameters (about 1% of the total model parameters), achieved a classification accuracy of 78.9% ± 1.7%. This performance was competitive but slightly inferior to a smaller model with 36M parameters, finetuned using traditional methods, which attained an accuracy of 80.4% ± 0.9%. The results suggest that while foundation models possess considerable potential, their efficacy in medium-sized datasets and in transitioning from single-view to multi-view image analysis, particularly where reasoning feature relationships across different mammographic views is crucial, can be challenging. This underscores the need for innovative transfer learning approaches to better adapt and generalize foundation models for the complex requirements of multi-view medical image analysis.
Fourier ptychography microscopy (FPM) is a computational imaging technique that enables high resolution and large FOV simultaneously. For FPM, multiplexed LED illumination can significantly improve the efficiency of image data acquisition at the cost of deteriorated quality in the reconstructed images. In this study, we aim to evaluate the imaging quality of multiplexed FPM with different illumination configurations. For this purpose, a prototype FPM microscope was developed, which was equipped with a 4×/0.1 NA objective lens. This prototype was used to test 1 LED conventional, 2 LED multiplexed, and 4 LED multiplexed FPM illumination configurations on a standard USAF 1951 resolution target and a cytology sample. Modulation transfer function (MTF) curves were generated from the reconstructed images to quantitatively compare the performance of different LED combinations. The results demonstrated that the resolution target image reconstructed using 1 LED illumination raw images can resolve up to 912.3 lp/mm, but it decreased to 812.7 lp/mm and 724.1 lp/mm when 2 LED and 4 LED illumination were adopted, respectively. The corresponding MTF curves indicate decreased contrast on most spatial frequencies when comparing reconstructed results between multiplexed (2/4 LED) and conventional illumination configurations. Accordingly, the quality of reconstructed clinical cytology sample images decreases as the number of LEDs per image increases. However, all of them have satisfactory quality for most clinical applications. This preliminary study provides useful information to facilitate the development of multiplexed illumination FPM imaging systems in the future.
Vasospasm is a common complication in aneurysmal subarachnoid hemorrhage (aSAH). Currently, patients with aSAH are usually monitored at intensive care unit (ICU) for approximately 14 days for early detection and treatment of vasospasm. To facilitate the diagnosis and decision-making process, this investigation aims to combine radiomics and deep learning technologies to predict vasospasm that requires intra-arterial treatment for patients with aSAH. For this purpose, a retrospective dataset was collected, containing a total of 52 aSAH patients. Next, a total of 1032 radiomic features and 768 vision transformer (ViT) based features were computed for each case to comprehensively quantify the aSAH characteristics. Based on the initial feature pool, analysis of variance (ANOVA) F1 score was applied to select 30 best performed features as the optimal feature cluster. Finally, a support vector machine (SVM) based classifier was trained to predict the vasospasm, and the model performance was evaluated using a 4-fold cross-validation strategy. Receiver operating characteristics (ROC) curve and confusion matrix were adopted as the assessing indices. The result show that the model achieved an area under the ROC curve (AUC) of 0.86±0.03, positive predictive value of 78%, negative predictive value of 76%, and overall accuracy of 77%, respectively. This investigation initially verified the feasibility of using CT images to accurately predict cerebral vasospasm.
Ovarian carcinoma is the most lethal malignancy in all kinds of gynecologic cancers, and radiomics based image marker is an effective tool for the early-stage prediction of the chemotherapies applied on ovarian cancer patients. This investigation aims to compare and evaluate the predicting performance of the 2D and 3D radiomics features. During the experiment, the tumors were first segmented from the CT slices, based on which a total of 1032 2D radiomics features and 1595 3D radiomics features were extracted. These features are related to tumor shape, density and texture properties. Next, a least absolute shrinkage and selection operator (LASSO) feature selection method was adopted to determine optimal features clusters for 2D and 3D feature pools respectively, which were used as the input of support vector machine (SVM) based prediction models. During the experiment, a total of 99 cases were selected from a previously established dataset at our medical center. The model performance was assessed by receiver operating characteristic (ROC) curve. The results indicated that the 2D and 3D feature based models achieved an area under the curve (AUC) of 0.85±0.03 and 0.89±0.02, while the overall accuracies were 0.76 and 0.81 respectively. These results indicate that the overall performance of the 3D feature is higher than the 2D features. But the sensitivity of the 2D model is higher at some certain specificity range. This study initially reveals the difference between the 2D and 3D features, which should be meaningful for the optimization of the radiomics based clinical decision support tools.
Fourier Ptychography Microscopy (FPM) is considered as one emerging technology for the development of high efficiency and low-cost microscopic scanners. One of the major advantages of FPM is its large depth of field (DOF), which significantly reduces the mechanical accuracy requirement of the scanning stages. In this study, we experimentally measured the DOF for our FPM prototype under different illumination conditions. The measurements were based on the theory that the DOF is considered as the range along optical axis for which the contrast is above 80% of the maximum when adjusting the focus location. Accordingly, the contrast is estimated using the bar pattern on the standard resolution target USAF1951 where the modulation transfer function (MTF) curve value drops to 0.5. During the experiment, the FPM prototype is equipped with a 4×/0.13 NA objective lens, and the DOF measurement was conducted with conventional single LED illumination and symmetric illumination. The results demonstrate that the DOF of the single LED illumination FPM is 15.3 µm, which is close to the DOF of the objective lens (14.5 µm). The DOF increases to 22.7 µm when symmetric illumination is adopted, which agrees with the theoretical conclusion. This investigation provides meaningful information for the future optimization of the FPM-based microscopic digitizers.
Metaphase chromosome karyotyping plays an important role in the diagnosis of certain cancers and some genetic diseases by detecting chromosome abnormalities. For this technique, high magnification objective lens is used to ensure the chromosome’s band pattern sharpness, but the small field of view (FOV) of the lens makes the imaging of chromosomes very tedious and time consuming. The purpose of this study is to verify the use of the Fourier ptychography microscopy (FPM) system in high-resolution karyotyping. Based on our former study, we further expanded the theoretical NA of the FPM system to 1.11 with a 20×/0.4 NA objective lens and higher illumination angles. To evaluate the resolving power of the FPM system, a 1951 USAF resolution target was imaged to create the modulation transfer function (MTF) curves. The performance of the FPM system was also assessed by imaging chromosomes acquired from blood and bone marrow pathological samples. The results were compared with a conventional 100×/1.45 NA oil immersion objective lens. The MTF curves demonstrate that the contrast of the FPM system is inferior but close to the 100× objective lens (1.45 NA). As compared to the images acquired by the 100×/1.45 NA oil immersion objective lens, the chromosome images recovered by the FPM system contain all the band patterns, despite the loss of some fine details. This study initially verified that the high NA FPM system can guarantee the sharpness of chromosome band patterns as the conventional high magnification oil immersion objective lens, while enabling a large FOV without the utilization of oil immersion medium.
The study aims to develop a novel computer-aided diagnosis (CAD) scheme for mammographic breast mass classification using semi-supervised learning. Although supervised deep learning has achieved huge success across various medical image analysis tasks, its success relies on large amounts of high-quality annotations, which can be challenging to acquire in practice. To overcome this limitation, we propose employing a semi-supervised method, i.e., virtual adversarial training (VAT), to leverage and learn useful information underlying in unlabeled data for better classification of breast masses. Accordingly, our VAT-based models have two types of losses, namely supervised and virtual adversarial losses. The former loss acts as in supervised classification, while the latter loss aims at enhancing the model’s robustness against virtual adversarial perturbation, thus improving model generalizability. To evaluate the performance of our VAT-based CAD scheme, we retrospectively assembled a total of 1024 breast mass images, with equal number of benign and malignant masses. A large CNN and a small CNN were used in this investigation, and both were trained with and without the adversarial loss. When the labeled ratios were 40% and 80%, VAT-based CNNs delivered the highest classification accuracy of 0.740±0.015 and 0.760±0.015, respectively. The experimental results suggest that the VAT-based CAD scheme can effectively utilize meaningful knowledge from unlabeled data to better classify mammographic breast mass images.
Significance: Searching analyzable metaphase chromosomes is a critical step for the diagnosis and treatment of leukemia patients, and the searching efficiency is limited by the difficulty that the conventional microscopic systems have in simultaneously achieving high resolution and a large field of view (FOV). However, this challenge can be addressed by Fourier ptychography microscopy (FPM) technology.
Aim: The purpose of this study is to investigate the feasibility of utilizing FPM to reconstruct high-resolution chromosome images.
Approach: An experimental FPM prototype, which was equipped with 4 × / 0.1 NA or 10 × / 0.25 NA objective lenses to achieve a theoretical equivalent NA of 0.48 and 0.63, respectively, was developed. Under these configurations, we first generated the system modulation transfer function (MTF) curves to assess the resolving power. Next, a group of analyzable metaphase chromosomes were imaged by the FPM system, which were acquired from the peripheral blood samples of the leukemia patients. The chromosome feature qualities were evaluated and compared with the results accomplished by the corresponding conventional microscopes.
Results: The MTF curve results indicate that the resolving power of the 4 × / 0.1 NA FPM system is equivalent and comparable to the 20 × / 0.4 NA conventional microscope, whereas the performance of the 10 × / 0.25 NA FPM system is close to the 60 × / 0.95 NA conventional microscope. When imaging the chromosomes, the feature qualities of the 4 × / 0.1 NA FPM system are comparable to the results under the conventional 20 × / 0.4 NA lens, whereas the feature qualities of the 10 × / 0.25 NA FPM system are better than the conventional 60 × / 0.95 NA lens and comparable to the conventional 100 × / 1.25 NA lens.
Conclusions: This study initially verified that it is feasible to utilize FPM to develop a high-resolution and wide-field chromosome sample scanner.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.