2.1.

Polarized HSI

In this study, we employ a customized polarized hyperspectral microscope, which is capable of full Stokes polarized hyperspectral imaging.³² The core components of the imaging system include an upright optical microscope with an integrated halogen light source, two linear polarizers, two liquid crystal variable retarders (LCVRs), and a customized SnapScan hyperspectral camera. The LCVRs and polarizers are for polarized light imaging. The transmissive axis of polarizer 1 was set at 45 deg, and the transmissive axis of polarizer 2 was at 0 deg. The fast axis of LCVR 1 was set at 0 deg, and that of LCVR 2 was at 45 deg. The SnapScan hyperspectral camera was able to acquire data through the translation of the imaging sensor inside of the camera. It had a wavelength range of 470 to 750 nm with 97 spectral bands and an image resolution of $1200\times 1200\text{pixels}$. The polarized light imaging components and HSI components worked together to acquire images in the visible-infrared wavelength range. Images were collected under $10\times$ magnification and $40\times$ magnification. Figure 1 demonstrates the setup of the imaging system with transmissive axis orientations of the polarizers and the fast axis orientations of LCVRs.

Fig. 1

The setup of the PHSI microscope system. The transmissive axis orientation of polarizer 1 was set at 45 deg, and that of polarizer 2 was set at 0 deg. The fast axis orientation of LCVR1 was set at 0 deg, and that of LCVR2 at 45 deg.

The system is capable of full Stokes polarimetric imaging by acquiring four element images, namely $I_h$, $I_v$, $I_{45}$, and $I_{rc}$. $I_h$ represents the light intensity measured with a horizontal linear analyzer, in which the retardations of LCVR 1 and LCVR 2 are both set to 0 rad; $I_v$ represents the light intensity measured with a vertical linear analyzer, in which LCVR 1 is set at 0 rad retardation and LCVR 2 is set at $\pi$ rad retardation; $I_{45}$ represents the light intensity measured with a 45 deg oriented linear analyzer, in which LCVR 1 and LCVR 2 are both set at $\pi /2$ rad retardation; and $I_{rc}$ represents the light intensity measured with a right circular analyzer, in which LCVR 1 is set at 0 rad retardation and LCVR 2 is set at $\pi /2$ rad retardation. The phase retardation of LCVR is determined by applying different values of voltage on it. In the PHSI dataset obtained by the system, each element image is a 3D data with two spatial dimensions and one spectral dimension. After image acquisition, the Stokes vectors ($S0$, $S1$, $S2$, and $S3$) were calculated using the four element images, as defined in Eq. (1). Each Stokes parameter was also a 3D data cube, as shown in Fig. 2. In addition, the value of $S0$ is equal to the value of total light intensity.

Fig. 2

Diagram of full-polarization Stokes vector data cubes, each of which has three dimensions including two spatial dimensions ($x$, $y$) and one spectral dimension ($\lambda$).

2.2.

Sample Preparation

Fresh surgical tissue samples were obtained from patients who underwent surgical resection of head and neck cancer, as we described earlier.³⁷ For each patient, a sample of the primary tumor, a normal tissue sample, and a sample at the tumor-normal margin were collected. Fresh ex vivo tissues were formalin fixed, paraffin embedded, sectioned, stained with H&E, and digitized using whole-slide scanner. Then, a board-certified pathologist specializing in head and neck cancer outlined the cancer margin in the digitized histology images. The annotations were used as the histologic ground truth in this study.

2.3.

Synthetic RGB Images

To generate synthetic RGB images from Stokes vector data cubes, we adopted an HSI-to-RGB transformation function similar to the spectral response of human eyes and modified it specifically for our PHSI data,^4,38 as shown in Fig. 3. In the transformation process, three spectral response curves (R, G, and B) multiply with the data cubes, respectively, to generate three channels (red, green, and blue) in the synthetic RGB images. We applied this HSI-to-RGB transformation to all four Stokes vector data cubes ($S0$, $S1$, $S2$, and $S3$) to generate four PHSI-synthesized RGB images. We used the synthetic RGB images of $S0$ as an H&E analog, which may not perfectly represent a standard visible light microscope.

Fig. 3

Transformation function to synthesize RGB images from Stokes vector data cubes. The weights for red (R), green (G), and blue (B) channels are modified specifically for our PHSI data.

2.4.

System Calibration

We implemented the calibration strategy based on a quarter wave plate following a standard calibration method.³⁹ We rotated the fast axis of the quarter wave plate from 0 deg to 180 deg with a 10-deg increment and acquired a set of four hyperspectral data cubes at each step. The theoretical relationships between the Stokes vector parameters ($S1$, $S2$, and $S3$) normalized by $S0$ and fast axis positions ($\theta$) are shown in Eq. (2):

We selected three wavelengths for calibration (470, 525, and 626 nm) and calculated the root mean square error (RMSE) for normalized $S1$, $S2$, and $S3$. The RMSE is defined as follows:

In this study, 470 nm is the starting wavelength of the hyperspectral camera. 525 nm has a 55 nm increment from 470 nm, and 626 nm has a 156 nm increment from 470 nm. The three chosen wavelengths cover a large wavelength range of the data cubes; thus we believe that they can be representative to show that our system can accurately generate Stokes vector parameters within the wavelength range.

2.5.

Contrast of Collagen

The metric that we introduce in this study to evaluate the imaging of collagen is contrast, which is determined by the difference in the color and brightness of the object and other objects within the same field of view (FOV). The calculation of contrast is expressed as

2.6.

Spectra Extraction

To accurately extract the spectra of Stokes vector related parameters from collagen fibers, normal squamous cells, and tumor cells, we manually outlined the regions of interest (ROIs) in the synthetic RGB images of $S0$ to generate the binary masks of collagen fibers, normal cells, and tumor cells. Because the FOVs of four Stokes vector cubes were identical, we applied the same binary masks to all four data cubes of Stokes vector parameters to extract the pixel-level spectra of the ROI and calculated the mean and standard deviation of all pixels in the ROI. This process helps to remove the influence of background pixels on the spectra of Stokes vector parameters.

3.1.

Calibration of the Polarized Hyperspectral Microscope

We carried out system calibration and calculated RMSE at three different wavelengths (470, 525, and 626 nm), as given in Table 1. All RMSE values were $<0.2$, which shows the satisfactory performance of our system. Figure 4 shows the theoretical curve and experimental data of the relationship between Stokes parameters ($S1$, $S2$, and $S3$) and fast axis positions at 525 nm. It can be seen that the experimental data agree with the theoretical curves of $S1$, $S2$, and $S3$.

Table 1

RMSE of Stokes vectors S1, S2, and S3 at 470, 525, and 626 nm, respectively.

Fig. 4

Calibration results of three Stokes vectors under 525 nm. The theoretical curves of $S1$, $S2$, and $S3$ and the fast axis positions are plotted along with the scattered plots of the experimental values of $S1$, $S2$, and $S3$ and fast axis positions: (a) calibration result of Stokes vector $S1$ with an RMSE of 0.169, (b) calibration of Stokes vector $S2$ with an RMSE of 0.155, and (c) calibration of Stokes vector $S3$ with an RMSE of 0.184.

3.2.

Enhanced Visualization of Collagen in Polarized Hyperspectral Images

In this section, we qualitatively and quantitatively demonstrate the enhanced visualization of collagen fibers in histological slides using PHSI. Figure 5 shows the PHSI-synthesized RGB images of two regions in normal tissues. Each synthetic RGB was generated using one Stokes vector parameter, and each set of four Stokes parameter images share the same FOV. The normal stroma tissue in Fig. 5(a) contains both dense and sparse collagen fibers. We found that dense collagen fibers, which are mostly located at the bottom region of the images, could be observed in all four Stokes vector parameters. In addition, the sparse collagen fibers in the upper side of the images were difficult to be recognized in $S0$, but they were easily recognized in $S1$, $S2$, and $S3$. Figure 5(b) shows another region in normal stroma tissue containing sparse collagen fibers. Similarly, it is hard to identify the regions containing sparse collagen fibers in $S0$, but the distribution of sparse collagen fibers can be seen clearly in $S1$, $S2$, and $S3$. Figure 6 shows the synthesized RGB images of the Stokes vector parameters of two cancerous tissue regions containing collagen fibers. Specifically, Fig. 6(a) shows a region where fibers accumulated within the tumor, and in Fig. 6(b) the fibers were around the tumor. Similar to the images in Fig. 5, it is difficult to identify the sparse collagen fibers in $S0$, but their distribution can be observed clearly in $S1$, $S2$, and $S3$.

Fig. 5

Synthesized RGB images generated using four Stokes vectors ($S0$, $S1$, $S2$, and $S3$) from two different regions in normal stroma tissue: (a) normal stroma containing both dense and sparse fibers and (b) normal stroma with sparse fibers.

Fig. 6

Synthesized RGB images of $S0$, $S1$, $S2$, and $S3$ from two cancerous regions containing fibers: (a) cancerous tissue with fibers accumulated in tumor and (b) cancerous tissue with fibers accumulated around tumor.

Contrast was calculated on all images with collagen shown in Figs. 5 and 6, and the values are given in Table 2. All images were saved in the double format, so the values of contrast are between 0 and 1. It can be seen that there is a significant increase of contrast on all images from $S0$ to $S1$, $S2$, and $S3$ in a sequence.

Table 2

Contrast for S0, S1, S2, and S3 of images with collagen under different conditions.

3.3.

Spectral Analysis of Collagen Using PHSI

To further investigate the usefulness of PHSI for collagen study, we extracted spectra of Stokes vector parameters of collagen and calculated their mean and standard deviation. Figure 7(a) shows the PHSI-synthesized RGB images of a small region in normal tissue containing dense collagen fibers as well as the corresponding spectra of four Stokes parameters $S0$, $S1$, $S2$, and $S3$. We find a low peak near 550 nm appearing in the spectra of all four Stokes parameters. In addition, there is a common high peak on the spectra of $S1$, $S2$, and $S3$ near 575 nm. Figure 7(b) shows the synthesized RGB images and spectra of Stokes vector parameters of a small normal tissue region containing sparse fibers. Similarly, we can see the low peaks at 550 nm in the spectra of $S0$, $S1$, $S2$, and $S3$ and the high peaks at 575 nm on the spectra of $S1$ and $S2$. However, as there are more fluctuations on the spectra of $S3$, the high peak at 575 nm is not very clear to be seen.

Fig. 7

Synthesized RGB and average spectra of Stokes vectors from a small region in normal tissue: (a) normal tissue containing dense fibers and (b) normal tissue containing sparse fibers.

Figure 8(a) shows the PHSI-synthesized RGB images and spectra of Stokes vector parameters of a small tumor tissue region containing fibers growing within the tumor. The spectra shape of $S0$, $S1$, and $S2$ are consistent with what we see in Fig. 7. For the spectra of $S3$, the low peak at 550 nm can be observed with more fluctuations. Figure 8(b) shows a small tumor region containing fibers growing around tumor cells. The spectra shapes of $S0$, $S1$, and $S2$ are consistent with what we find in the previous figures. For $S3$, we can observe a more significant increase after 550 nm. Figure 9 demonstrates the combination of the spectra ($S0$, $S1$, $S2$, and $S3$) in Figs. 7 and 8.

Fig. 8

Synthesized RGB and average spectra of Stokes vectors from a small region in cancerous tissue: (a) tumor tissue with fibers growing within and (b) fibers around the tumor.

Fig. 9

Spectral comparison of different fibers in Figs. 7 and 8.

3.4.

PHSI-Enhanced Visualization of Normal and Tumor Cells

Figure 10(a) demonstrates the synthetic RGB images of Stokes vector parameters ($S0$, $S1$, $S2$, and $S3$) from a normal tissue region. We find that $S1$ and $S2$ enhance the image signal from the connective tissue, and $S3$ enhances the image signal from the normal cells. The $50\mu \mathrm{m}$ scale bar is specified on $S0$. Figure 10(b) demonstrates the synthetic RGB images of Stokes vector parameters ($S0$, $S1$, $S2$, and $S3$) from a tumor tissue region. This piece of tumor tissue contains a number of not well-differentiated tumor cells, featured by irregular or elliptical nuclei with coarse heterochromatin aggregates, increased fragmentation, or budding of nuclei.^40,41 We find that $S1$, $S2$, and $S3$ help to improve the visualization of tumor cell morphology.

Fig. 10

Synthesized RGB images generated from Stokes vectors of normal and tumor tissues: (a) normal tissue from a normal slide and (b) cancerous tissue from a tumor slide.

3.5.

PHSI Spectra of Normal and Tumor Cells of HNSCC

From the images acquired with our customized polarized hyperspectral microscope, we extracted the spectra of Stokes vector parameters of normal and tumor cells and calculated the mean and standard deviation. Figure 11 shows the PHSI-synthesized RGB images of Stokes vector parameters of two typical normal cells with a good differentiation of nucleus and cytoplasm structures, as well as the corresponding spectral curves of $S0$, $S1$, $S2$, and $S3$. The morphology of normal cell could be visualized clearly in all synthesized RGB images but is enhanced in $S1$ and $S2$. For the spectra of $S0$, a low peak near 550 nm is observed on both Figs. 11(a) and 11(b). For the spectra of $S1$, two close high peaks are observed near 475 nm and near 550 nm. For the spectra of $S2$, we observe a number of fluctuations, and the maximum normalized intensity appears near 550 nm. For the spectra of $S3$, there are also a number of fluctuations, with the minimum of normalized intensity appearing near 550 nm. Figure 12 demonstrates the PHSI-synthesized RGB images of Stokes vector parameters of two tumor cells and the corresponding spectra of $S0$, $S1$, $S2$, and $S3$. From Fig. 12(a), we find that it is hard to see the multiple nucleoli in $S0$, but we observe a good image contrast and morphological details of the nucleoli in $S1$, $S2$, and $S3$. The tumor cell in Fig. 12(b) is under a condition of increasing fragmentation of nuclei. It can be seen that the PHSI-synthesized RGB images of $S1$, $S2$, and $S3$ improve the visualization of nuclei fragmentation. The shape of the Stokes parameter spectra of two cells in Fig. 12 are consistent with each other. For $S0$, a low peak near 550 nm is observed. For $S1$, the two close high peaks near 475 nm and near 550 nm disappear. For $S2$, the two close high peaks near 475 and 550 nm tend to appear. For $S3$, there are several fluctuations. Figure 13 demonstrates the spectra comparison among two normal cells and two SCC cells. We find that the overlapping in the spectra of $S0$ and $S2$ is large among these cells. However, the spectra of $S1$ and $S3$ of the two SCC tumor cells do not overlap with those of the two normal cells, thus making the spectra of normal cells and tumor cells easier to be differentiated.

Fig. 11

Synthesized RGB and Stokes vector spectra of single normal cell. In both (a) and (b), the spectra of four Stokes vectors show differences, and the nucleoli position is clearly observed.

Fig. 12

Synthetic RGB and Stokes vector spectra of tumor cells. In both (a) and (b), the spectra of four Stokes vectors show differences, and the nuclei morphology is clearly observed.

Fig. 13

Spectra comparison between the selected two normal cells and two tumor cells on the HNSCC slides.

Furthermore, we implemented two-sample $t$ test on the average spectra of Stokes vector parameters of 15 normal cells and 15 tumor cells, which were selected from three different patients (five normal cells and five tumor cells from each patient). The $p$ values of different bands in each Stokes parameter are given in Fig. 14. It can be seen that the $p$ values of $S2$ and $S3$ significantly decrease compared with those of $S0$, and $S3$ has $p$ values $<0.05$ at most wavelengths. Note that the selection of these 30 cells did not strictly follow the assumption in standard statistical tests of statistically independent samples.

Fig. 14

$P$ values based on the two-sample $t$ test among the average spectra of 15 normal cells and 15 tumor cells.