1.

INTRODUCTION

Chest radiography is widely used for the diagnosis of lung diseases. It has a high spatial resolution, but being a projection image, tissues are shown overlapped without depth information and with a masking of low-density structures behind the bones. This limitation has been partially solved by tomosynthesis, a technique mainly used in mammography, but it still cannot convey density information. This is a major limitation for the use of tomosynthesis in chest imaging, due to the wide range of pathologies with increased lung parenchymal density. The only available technique to retrieve a density map is computed tomography (CT), but its use is limited by issues of availability and/or radiation dose control.

Several reconstruction strategies have been proposed to obtain an improved reconstruction of tomosynthesis data. In,¹ the authors proposed a Digital Breast Tomosynthesis (DBT) reconstruction based on the filtered back-projection (FBP) algorithm by a modification of the ramp filter, while^{2, 3} propose iterative reconstruction (IR) algorithms. Nevertheless, although these approaches increase image quality, the results shown are still far from the quantitative information of a CT.

During the last few years, image reconstruction methods have incorporated Deep Learning strategies as: (1) a post-processing step, after reconstruction,⁴ (2) a pre-processing of the projection data,⁵ or (3) within an IR algorithm, in the so-called unrolled methods.^{6, 7} Nevertheless, despite further improving image quality over traditional IR, none of the proposed methods show true tomographic results.

In this work, we propose a reconstruction methodology to obtain tomographic images (close to a CT) from chest tomosynthesis. The methodology consists of an FDK-based reconstruction step followed by post-processing based on Deep Learning.

2.

DATABASE GENERATION

The database consists of tomosynthesis reconstructions as inputs and CT scans as a reference. We used 55 chest CT studies, 12 provided by the Hospital General Universitario Gregorio Marañón and 43 extracted from the Medical Imaging and Data Resource Center (MIDRC) portal from three different datasets. All studies were interpolated to a common isotropic voxel size, 1.25×1.25×1.25 mm, which is larger than that of any of the original CT volumes.

To simulate the tomosynthesis projections and perform the reconstructions, we used FuxSim,⁸ a software that allows simulating different acquisition geometries based on a density map and system parameters. The simulations were based on a standard chest tomosynthesis protocol: 61 projections with a total source-to-detector distance of 1800 mm and a scan angle of 30 degrees. The resulting projections had a matrix size of 1072×1072 pixels with a pixel size of 0.4×0.4 mm. Since the patient is positioned next to the detector in the acquisition of chest tomosynthesis data, we cropped the volumes to remove the excess of air from the CT field of view and eliminated the bed and other patient supports, as shown in Fig. 1.

Figure 1.

Pre-processing of CT data to simulate patient position in tomosynthesis system: removal of the bed, patient support and excess of air.

To avoid truncation artifacts in the tomosynthesis projections, caused by the spatial limitation of the chest CT scans in the vertical direction, we extended the CT volumes as shown in Fig. 2.

Figure 2.

Tomosynthesis projections from CT for three different positions before (top) and after (bottom) extension. White arrow shows truncation artifact.

The simulated tomosynthesis data were reconstructed with an FDK-based algorithm, modifying the minimum value of the ramp filter to avoid the loss of mean value. For each study, we obtained a volume of 275×192×99 corresponding to the central part of the thorax, which is the ROI evaluated for diagnosis, with 1.25×1.25×1.25 mm voxel size. Fig. 3 shows the central coronal slice of the tomosynthesis reconstruction together with that of the reference CT for three of the volumes.

Figure 3.

Coronal slices of the result of FDK-based reconstruction from the tomosynthesis data (top) and their corresponding CT (bottom).

The database was divided into the following sets:

3.

POST-PROCESSING STEP

The post-processing step has as input the FDK-based reconstruction and as output a volume with enhanced tomographic information. The proposed architecture is a modification of the original version of the U-Net architecture⁹ to include new strategies recently proposed in the literature. The encoder was replaced by ResNet- 34,¹⁰ due to the improved performance offered by the residual blocks over the original version of the U-Net encoder. The decoder consists mainly of oversampling blocks composed of a convolutional layer, followed by an average pooling layer (APL), a batch normalization (BN) layer, a ReLU, and two convolution layers (Conv2D) followed by a ReLU each. At the output of the first convolutional layer, the pixel shuffle operation with ICNR initialization¹¹ is applied. The complete network is shown in Fig. 4.

Figure 4.

Proposed U-net network architecture.

For the initialization of the encoder, we used the pre-trained version of Resnet-34 for Imagenet.¹² The decoder was initialized randomly adapting it to the output of our problem. The training methodology was based on Transfer Learning,¹³ first freezing the encoder to train only the decoder and then training the end-to-end network for fine-tuning.

Perceptual Loss (PL)¹⁴ was chosen as the cost function to assure the recovery of fine details. We chose Adam¹⁵ as the optimizer, due to its higher convergence speed, and a weight decay equal to 10^‒3 as regularization strategy. To determine the learning rate, we used the test presented by Leslie N. Smith,¹⁶ which resulted in an optimal learning rate of 10^‒3 for decoder training and 10^‒5 for end-to-end network training.

Although the model processes 2D images, we incorporate 3D information by alternating coronal and axial slices in a multi-stage training strategy. As shown in Fig. 5, in the first stage we trained with the coronal view of the FDK-based reconstruction as inputs and the coronal CT view as target. In the next round, the coronal prediction of the first stage was resliced to obtain the axial view, this being the input at this stage and the target being the CT axial view. The axial view of the new prediction was then resliced to obtain the coronal view again and a new training was performed.

Figure 5.

Scheme of the training strategy.

4.

RESULTS

Fig. 6 shows a good recovery of general structure and realistic texture with the proposed reconstruction methodology, especially in areas with well-defined in the FDK-based reconstruction, such as the spine.

Figure 6.

Coronal slices of: FDK-based reconstruction (left), prediction (center) and CT (right).

Fig. 7 shows the mean of the root mean square error (RMSE) with respect to the target CT for the six test volumes. We can see that our proposed methodology reduces the RMSE compared to FDK-based reconstruction in whole volume, but greater in the slices with tissues with higher contrast, such as spine and ribs (see coronal slices shown).

Figure 7.

Mean value of RMSE for the six test volumes along the different coronal slices with three FDK-based reconstruction examples.

Fig. 8 show a zoom of coronal slices in Figure 6, where we can see a blurring, and more importantly, structures that do not exactly match those in the original CT (see white arrows).

Figure 8.

Zoomed ROIs from coronal slices in Fig. 6: FDK-based reconstruction (left), prediction (center) and CT (right). White arrow highlights structures wrongly.

Fig. 9 shows the predicted image after each stage. After the first stage, we can still see artifacts in the axial view from the FDK-based reconstruction, because the prediction is done on the coronal view and the FDK. The second stage, working on the axial view, these artifacts are removed, the contours are recovered, and texture is improved. The last stage further improves texture improves without altering shapes already found in previous stages.

Figure 9.

Coronal (top) and axial (bottom) slices of: FDK-based reconstruction (A), the first (B), second (C) and third (D) stages.

5.

DISCUSSION

We have presented a reconstruction methodology for chest tomosynthesis based on FDK followed by post-processing based on deep learning. We use the transfer learning workflow, with U-Net architecture having a ResNet34 as encoder. Although our network processes 2D images, we incorporate 3D information by alternating coronal and axial slices in a multi-stage training strategy. The first stage of training uses the coronal view because it is the one with most information in FDK-based reconstruction. The evaluation shows a good reconstruction, especially in areas with high contrast and well-defined structures in the FDK-based reconstruction, such as the spine, and a very realistic tomographic texture. However, in the predicted images, we can find structures that do not match exactly with the original CT (biases). Looking at the result of the different stages, we can see that the biases appear in the first stage and are maintained through subsequent stages. Future work will evaluate the combination of both prediction of previous stage together with the original FDK-based reconstruction at the input of the second and third stages to minimize the propagation of bias along the stages.

In this work we use a voxel size bigger than that of the original CTs due to computational limitations. This might be together with the subsampling layers of our U-Net might be responsible of the loss of spatial resolution. Future work will try to overcome the memory limitation by distributed training in using multiple GPUs.