2.1.

Overview of Several Approved FDA Systems and Contrast Agents

Multiple FGS devices have been cleared for clinical use under the current FDA 510(k) pipeline. Figure 1 highlights examples of FDA cleared FGS devices, and 510(k) numbers are provided as a guide to the approved uses of each system. As described in the highlighted 510(k) summary of each device, common indications of FGS systems include visualization of blood-flow and related tissue perfusion, as well as related tissue-transfer in free flaps during micro and reconstruction surgeries. Nevertheless, clinical trials have explored numerous uses that might be particular to individual devices, some of them being encompassed within 510(k) indications of use. For example, in gallbladder removal surgery (laparoscopic cholecystectomy) the RUBINA™ guided surgery system combined with an ICG contrast agent has helped to identify biliary and vascular tissues, as well as the presence of abnormalities that can cause complications during surgery.¹³

Fig. 1

Examples of several FDA cleared systems. 510(k) clearance numbers are also provided for further understanding of approved indications for use. Systems displaying an asterisk (*) were used in this work for evaluation of the proposed standardization procedure.

The PINPOINT system has been used together with an ICG contrast agent for identification and resection of liver hepatoblastomas that otherwise would not be detected, hence improving the patient survival rate post-surgery.¹⁴ The use of DaVinci Firefly,¹⁵ in which robotic gastrectomy (lymphadenectomy) can be performed in conjunction with an ICG contrast agent, showed that lymph nodes were better localized through FGS than through the traditional surgical methods. FGS platforms have also aided in the visualization of intestinal perfusion. In a case of mesenteric ischemia, the Elevision system was used to estimate which bowel tissue to resect during surgery.¹⁶ Real time cystectomy has also been performed through fluorescence guidance by the SPY-PHI system.¹⁷ Another important application is breast cancer removal and reconstruction surgeries in which FGS systems such as OnLume Avata¹⁸ and VITOM-ICG¹⁹ have improved surgery outcomes. In the case of the OnLume Avata FGS system, adaptations on its design have allowed for surgery under room light conditions. The VITOM-ICG setup has also been utilized in neurosurgery²⁰ for 3D exoscopic visualization of cranial and spinal tissues. Even though ICG is the only major contrast agent approved for use in these FGS devices, multiple evolving contrast agents are of future interest. Beyond ICG, there are several system developments aimed at other contrasts or endogenous fluorescence signals. Devices such as Fluobeam Clinical²¹ and Fluobeam LX²¹ have been used to image the near-infrared (NIR) autofluorescence of thyroid/parathyroid glands, which can be excited around 785 nm with emission peaks from 820 to 830 nm. Of importance, these emission and excitation profiles overlap with those of ICG. Another contrast agent explored is 5-ALA, which promotes intracellular generation of protoporphyrin IX (PpIX) in cancer cells.²² Imaging of PpIX has been widely deployed in neurosurgery with the approved formulation of Gleolan²³ and in the bladder with the approved formulation Cysview.²⁴ As described by the review works of Pogue et al.²⁵ and Hadjipanayis,^23,26 5-ALA induced PPIX guided surgery is one of the great successes of FGS. Surgical microscopes such as the Leica FL400 (Leica, Illinois, United States) and Zeiss BLUE 400/OPMI Pentero 900²⁷ have improved resection of abnormal brain tissues such as high grade gliomas, significantly changing patient outcomes. Even though this standardization work is limited to devices that allow for macroscopic open-field imaging rather than surgical microscopes due to differences in their optical arrangement, a similar standardization workflow could be implemented by accounting for the size of the microscopic field of view (FOV) in the used characterization phantoms. PPIX surgical microscopes (e.g., Leica and Zeiss) often have configurations for ICG imaging; hence with appropriate ICG phantoms that are scaled to their microscopic FOV, the proposed characterization procedure can be used. In addition to being limited to macroscopic open-field FGS systems, the current pipeline is also limited to systems that operate with ICG or where the contrast agent has overlapping excitation and emission profiles to those of ICG (e.g., parathyroid glands autofluorescence). This was due to the unavailability of photo-stable phantoms for contrasts different than ICG (e.g., PPIX). A similar characterization approach can be followed for FGS devices with contrast agents different than ICG in which photo-stability of the agent can be validated.

Methylene blue²⁸ and fluorescein are also FDA approved agents. Pafolacianine, which is linked to an NIR dye with a peak excitation and emission at 776 and 796 nm, respectively, and contains ligands to folate receptor-alpha that are over-expressed in multiple types of cancer (e.g., ovarian and lung cancer), was cleared by the FDA in 2021.²⁹ Examples of systems now aiming to be approved for pafolacianine detection are the EleVision IR, Quest Spectrum, and Explorer Air II systems.³⁰ Other dyes that have been applied in clinical studies are Cy5, Cy7, IRDye800CW, SO456, IRDye 700DX, SGM-101, and ZW800-I.³¹ Despite all of this activity, ICG imaging accounts by far for the largest use of FGS in open surgery today and likely for the near future. Thus, throughout this standardization procedure analysis, ICG phantoms were used. Specially developed target NIR contrasts that are excited and emit at similar bands to ICG can also be characterized.⁷ Within all of the available FDA cleared systems, this work focuses on the analysis of the proposed standardization procedure for a subset of five FDA cleared devices: Fluobeam Clinical, Fluobeam LX, SPY-PHI, OnLume Avata, and EleVision systems.

2.2.

Common Open-Field System Components and Quantitative Parameters

The selection of a contrast agent (exogenous or endogenous) is done with the development or choice of instrumentation that allows for its detection in the target tissue, even when the excited fluorescence is dimmer than the background intensity.⁵ A contrast agent can have a high quantum yield and high photo-stability, but the ability to detect its fluorescence and accurately resolve the pertinent fluorescent features is highly dependent on the FGS system components. In simplistic terms, an FGS system is composed of four main blocks, which are highlighted in Fig. 2 and are referred to throughout this manuscript as (i) illumination, (ii) detection, (iii) image processing, and (iv) user control blocks. One of the goals of the proposed standardization procedure is to quantify the performance of each of these blocks, except for the user control block in which only a description of useful features can be done. It is inevitable that most tests reveal a mixture of the influence of several of these system blocks; however, some are more definitive than others, as is highlighted below.

Fig. 2

Main building blocks of a generic fluorescence guided surgery (FGS) system. (a) Illumination block containing excitation and visual aid illumination sources. (b) Detection block containing, here, three different detectors as an example, with two for fluorescence and one for white light images; relay lenses (RL); optical filters (F); and dichroic mirrors (D1 and D2).^5,32 (c) Processing block in charge of image processing and rendering. Example figures displayed from Lu et al.³³ with author permission. (d) Depiction of user interface for manipulation of image display options.

2.2.1.

Illumination block

The illumination block [Fig. 2(a)] is composed of an excitation source matching the absorption wavelength of the contrast agent, while avoiding the emission band to be detected and in many cases a white-light source, which is needed for the surgeon to properly see the surgical field. Important parameters of the excitation source are the optical radiance or flux density $[{\mathrm{mW}/\mathrm{cm}}^2]$, bandwidth [nm], homogeneity of illumination, and optical-tissue penetration depth. The excitation source power density must be high enough to excite the fluorophore without causing it to photo-bleach but low enough to not harm the patient. Due to safety and ease of use requirements, optical irradiance for FGS systems typically ranges from 10 to $25\mathrm{mW}/{\mathrm{cm}}^2$ at the FOV.² Another important characteristic is the bandwidth of the excitation source. Typically with a Gaussian shape, as shown in Fig. 3(a), the excitation band is characterized by its full width half maximum (FWHM),³⁴ in which a narrower peak and smaller FWHM are indicative of high wavelength specificity. Hence, a larger FWHM provides a broader spectral output, which might result in excitation of other endogenous components; although in most diode laser based systems, this spectral band is quite narrow. But the spectral band is of special concern for visible regime wavelengths in which autofluorophores such as FAD, NADH, ribovaflin, lipofuscin, or melanin are excitable. ³⁵

Fig. 3

Summary of quantitative parameters tested with specific phantoms or measures. (a) Depiction of FWHM for an excitation band. (b) Graphic description of FOV and illumination power per unit area. (c) Graphic description of FOV uniformity and phantom used for characterizing it. (d) Comparison of systems with and without distortion and sample that can be used for characterization. (e) USAF 1951 fluorescent target that can be used for measuring resolution. (f) DOF depiction of fluorescent target and its CTF across different distances from the ideal focal plane or manufacturer defined WD. (g) Imaging phantom used for measurement of concentration sensitivity. (h) Imaging phantom used for measurement of depth sensitivity and expected trend.

Over the years, FGS systems have shifted into the NIR and NIR II regimes due to reduced autofluorescence bleed-through and scattering, hence avoiding major tissue autofluorophores and increasing imaging depth through tissue. There are cases in which contrast multiplexing is explored using a wider bandwidth, but most FGS systems aim at specific narrow excitation bands in the NIR to excite fluorescence of one contrast agent, with minimal leakage of this light into the detection band of the cameras. Laser diodes are the preferred method of excitation; however the use of light emitting diode (LED) sources has increased significantly, despite their broader spectral output, due to lower regulatory barriers in their use.⁵ Despite having to account for a loss in excitation power, LED based illumination blocks often add filters to provide higher specificity to the wavelength of interest. Spectrophotometer devices can collect the spectral distribution of the illumination source and FWHM values can be further retrieved. The use of pulsed illumination can also be implemented on this block. Pulsed illumination can then be combined with gated detection to temporally isolate signals from the ambient background. Some developing experimental systems use ultra-fast sampling to calculate nanosecond fluorescence lifetime values, which can be indicative of tissue environmental changes.^36,37 Optical irradiance or flux density³⁸ can be quantified through optical power meters that cover a mW/unit area range. In an ideal flat scenario in which an illumination source homogeneously covers the FOV, the optical power density is the same across unit area. Lenses and diffusers are often used to homogeneously distribute the excitation source in the FOV despite changes in source positioning. Knowing the illumination distribution of an FGS source with respect to the detector can help surgeons understand how the positioning of the instrument can account for topography differences in the patient FOV.

2.3.

Assessment of FGS System Parameters: Phantoms and Reference Targets

Fluorescence phantoms and reference targets are commonly used for FGS system performance assessments and comparisons. Over the past decade, the field of fluorescence phantoms has witnessed continuous development, with a transition from liquid and turbid phantoms to solid phantoms as the preferred reference for performance assessment.⁴⁴ Liquid phantoms have traditionally relied on the use of lipid emulsions and blood to mimic tissue scattering and absorption. The primary drawback of liquid and turbid phantoms is their short shelf-life (hours to days), which leads to single-use functionality, being prone to human preparation error, and reproducibility issues. By contrast, solid phantoms based on polyurethanes, silicone, and plastics generally provide long-term stability (months to years).¹¹ Recently, the use of cured two-part polyurethanes and 3D printed photocurable resins has enabled the manufacturing of long-term stable targets with tissue mimicking absorption and reduced scattering coefficients.⁴⁵ To simulate the fluorescence emission of fluorescence contrast agents, both quantum dots and laser dyes have been used. Quantum dots provide the advantage of photostability with tunable emission wavelengths along with the drawbacks of broadband visible-NIR excitation and high costs.⁹ By contrast, laser dyes can mimic both the emission and excitation of the fluorescence contrast agents with the drawback of inherent photobleaching due to the nature of being organic dyes. An ideal dye might provide an avenue for balancing the drawbacks of both quantum dots and laser dyes, but no viable candidates have been identified in the literature for current FGS applications. To perform the tests suggested in the recent consensus report,¹² evaluation of FGS systems, various tools, phantoms, and reference targets are needed. Figure 3 provides a summary of the qualitative parameters that are tested as part of the FGS system assessment. In brief, the assessment focuses on excitation illumination quantification (wavelength, power), image resolution, uniformity, and depth of field and contrast agent concentration sensitivity, linearity, and depth sensitivity.

The excitation spectra and FOV irradiance [Figs. 3(a) and 3(b)] are characterized using a spectrometer and optical power meter, respectively. FOV uniformity of the illumination-detection blocks can be initially verified through a flat and homogeneous fluorescent surface in vitro, as seen in Fig. 3(c), before moving to patient topography. Distortion testing can be performed with a grid of equidistant points or lines, as shown in Fig. 3(d). Image sharpness of imaging systems can be quantified through fluorescent spatial resolution targets (e.g., 1951 USAF target), such as the one displayed in Fig. 3(e), which contain varying spatial frequencies and size groups.⁴⁶ To know the DOF of an FGS system, a similar fluorescent resolution target to the one in Fig. 3(e) can be used and measured at closer and further distances than the plane of focus or WD. As described in Fig. 3(f), in an ideal case, the image contrast would be the same across all distances; however this is not true for any imaging system, so knowing the “in focus” range is important for surgeons to better utilize FGS devices. For the characterization of sensitivity, a phantom similar to the one shown in Fig. 3(g), in which fluorophore concentrations are varied within media that simulates the optical properties of tissue, can be used. In addition to knowing which is the lowest intensity detectable by the system, understanding the depth at which a signal can be reliably detected is of importance. Even when the use of NIR and NIR II wavelengths helps to increase the imaging depth for optical imaging setups, optical arrangements have limited tissue penetration depth compared with other imaging modalities due to the turbid nature and optical properties of tissue. Therefore, understanding an FGS system imaging depth at a determined concentration is important and can be done through a phantom such as the one displayed in Fig. 3(h), where the depth variations are induced by increasing the depth of tissue-like material under which the fluorophore is embedded. Phantoms such as these are also appropriate for margin assessment and signal to background ratios, in which a well delineated and low background image in respect to the fluorophore regions is desired to produce a higher contrast. The use of these phantoms with a simplified geometry provides well-defined optical parameters for system assessments. Anthropomorphic phantoms that mimic in-vivo conditions could be further used to provide application-specific performance metrics for characterization and comparison.

For this study, the commercially available reference targets (QUEL Imaging, New Hampshire, United States) are made from 3D-printed material that incorporate fluorophores, scatterers, and absorbers, as described in Ref. 47. The main material is a fluorescent photocurable resin with tunable optical properties. To create the resins, first a base photopolymer resin is selected, and then pre-dispersed fluorophores, absorbers, and scatterers are incorporated. Resins designed for liquid crystal display (LCD), masked stereolithography (MSLA) and digital light processing 3D printers with a curing wavelength of 405 nm were used. For pre-dispersion, stock solutions of the additives were prepared in dimethyl sulfoxide. Stock solutions can vary in concentration and amount to define the optical properties of the 3DP resin. The 3D models to be printed are then designed for each phantom and printed through an MSLA printer that uses an inverted lithography arrangement with an LED matrix (405 nm) and LCD screen for curing. For 3D printing, the exposure times and layer height range from 2.0 to 15.0 s and 0.01 to 0.10 mm, respectively.

3.1.

Tests 1 and 2 - Excitation Bands and Illumination Power Density

The spectral distribution of the excitation light was measured through a FLARE Miniature Spectrometer (Ocean Insight, Florida, United States) and fiber QP400-2-UV-BX (300 nm to $1.1\mu \mathrm{m}$). Neutral density filter ND-05A (Thorlabs, New Jersey, United States) was used to avoid signal saturation. To avoid spectral bleed-through, a dark fabric was used to cover the illumination source head and setup.

The acquired spectral distribution of the excitation sources is shown in Fig. 5(a) per each system. Of note, each of these systems has been cleared for ICG perfusion, as shown in Fig. S4 in the Supplementary Material, which gives the FDA clearance codes. The excitation and emission profiles of ICG were overlayed for visualization purposes. The FWHM of the spectral profiles, which is illustrated in Fig. 3(a) and values displayed in Fig. 5, were calculated in relation to wavelength units (nm). Results show FWHM values between 2.80 and 3.02 nm.

Fig. 5

Spectral bandwidths of excitation sources for systems 1 to 5. These excitation spectra were measured per FGS system through a FLARE spectrophotometer device. FWHM values were calculated per system. Absorption and emission profiles of ICG are also displayed for reference. Spectra were normalized to their maximum value for display.

Subsequently, optical power density was measured through power meter device 843-R (Newport, New Jersey, United States) and silicon photodetector 918D-SL-OD3R (Newport, New Jersey, United States). The calculated wavelength maximum was calibrated into the power meter, the photodetector was placed in the center of the FOV, and power was measured at the suggested WD. The estimated optical power densities of systems 1 to 5 together with important paramaters such as the estimated FOV size, manufacturer recommended WD, and reported excitation wavelengths are shown in Fig. 6.

Fig. 6

Summary of reported wavelength peaks and measured FOV sizes, used WD, and power densities for evaluation of FGS systems 1 to 5. Additional specifications are provided in the Supplementary Material.

3.2.

Test 3 - Spatial Resolution and Sharpness

To characterize the spatial resolution range, an ICG based resolution target, Resolution USAF 1951 (QUEL Imaging LLC), was used [Fig. 3(e)]. As shown in Fig. 7(a), the target has the largest feature size at $1.0\mathrm{lp}/\mathrm{mm}$ or $500\mu \mathrm{m}/\text{line}$ in group 0—element 1 and the smallest feature size at group 7—element 6 with $228.1\mathrm{lp}/\mathrm{mm}$ or $2.19\mu \mathrm{m}/\text{line}$.

Fig. 7

Spatial resolution test target (a) lp/mm used for quantification of spatial resolution and sharpness. (b) Images of resolution USAF 1951 fluorescent target as acquired with systems 1 to 5. (c) CTF results per selected group elements across the different systems. Systems 3 and 5 have high magnification (M) features, so results acquired with high M modes are also displayed for comparison purposes.

Before using the USAF fluorescent target, the systems were placed in the “focus plane” at the above described WDs with the aid of a “coin” target [Fig. S1(a) in the Supplementary Material]. The target has a 2 cm thickness ($z$-plane), which is the same as the rest of the targets that were used throughout the protocol. Since focusing systems to the appropriate focal plane might take several minutes, an extra target is used to avoid photobleaching effects on the phantoms due to external light sources. To understand whether samples are photobleaching as they are used, a radiometric target from QUEL Imaging LLC, displayed in Fig. S1(b) in the Supplementary Material, is used as a control, and voltage values are recorded with each use of the phantoms. The phantoms contain ICG fluorescent mimicking material with matching ICG excitation and emission profiles.⁴⁵ The material has an absorption coefficient (${\mu }_a$) of $\sim 0.018{\mathrm{mm}}^{-1}$ and reduced scattering coefficient (${\mu }_{s^{\prime }}$) of $\sim 1.4{\mathrm{mm}}^{-1}$ at 800 nm. The acquired images per each system are displayed in Fig. 7(b). For data acquisition and as summarized in Fig. S4 in the Supplementary Material, systems 1, 3, 4, and 5 allow for saving individual frames in “tiff,” “tif,” “jpg,” and “png” formats, respectively. For these systems, multiple individual frames were acquired per sample, and a single frame with a position of 1/totalframes was used for quantification. In the case of system 2, one of the acquired video frames (mp4 format) with position 1/totalframes was used. Systems 3 and 5 have magnification features, so an image using one of these magnifications was also displayed for comparison purposes. The respective bit depth and pixel resolution values were also displayed within the different images in Fig. 7(b). The features area of the target measures $\sim 1.5\times 1.5\mathrm{cm}$, and the % of space occupied in relation to the full FOV is also estimated. To quantify the spatial resolution and sharpness, the elements with 1.0, 1.26, 1.59, 2.0, 2.52, 3.17, 4.0, 5.04, and $6.35\mathrm{lp}/\mathrm{mm}$ were considered. These elements are also highlighted in Fig. 7(a). Per each acquired image, the background noise was thresholded with values between 0.1 and 0.4 of the maximum value. For each of them, a contrast transfer function (CTF %) value was calculated through Eq. (1), where $r$ is the selected lp/mm region. Instead of considering a single maximum and minimum value, the min and max values represent the average of 20% of the highest and 20% of the lowest values in the defined lp/mm region, respectively.

For $1.0\mathrm{lp}/\mathrm{mm}$, the image sharpness quantified through CTF (%) was above 95% for all systems. Of note, this is at the recommended WD and plane of focus. As shown in Fig. 7(c), as the spatial resolution of the target increases, the contrast decreases toward the lowest resolution of $6.35\mathrm{lp}/\mathrm{mm}$. For systems 2 and 5, the CTF% decreases below 60% at $1.26\mathrm{lp}/\mathrm{mm}$, below 20% at $2.52\mathrm{lp}/\mathrm{mm}$ and below 10% at the lowest resolution of $6.35\mathrm{lp}/\mathrm{mm}$. This was also true for system 3 without the use of the magnification feature. The highest variation between systems happens between 1.2 to $2.52\mathrm{lp}/\mathrm{mm}$. Of interest, systems that stayed with CTF(%) higher than 90% as the lp/mm increased were those that used magnification features and System 1 in which the sample was $\sim 3.4\%$ of the FOV.

3.3.

Test 4 - FOV Uniformity

To calculate the uniformity of the FOV, a phantom with homogeneous distribution was placed at the WD of the systems. The phantom was of size $\sim 18\times 18\mathrm{cm}$, which was larger than the targeted FOVs as depicted in Fig. 8(a). Images were acquired with parameters shown in Figs. S4 and S5 in the Supplementary Material per system. The used images are shown in Fig. 8(b). For each acquired image, the middle position of the $Y$ pixel dimension was calculated. Then 50 pixels above and below were selected, and the mean was calculated per pixel position of the $X$ axis. These values were then plotted as shown in Fig. 8(c) and used to estimate the homogeneity of the FOV. Since images have different $x$ and $y$ pixel dimensions, for display purposes, both the $x$ and $y$ axess were normalized. Results for system 4 show values between 0.95 and 1 for all pixel intensities, which means it shows a high level of homogeneity across the full FOV. System 5 displays intensities above 0.95 for values that fell between the first 75% of horizontal pixels and then values that decayed below 0.9 at the right-most border of the FOV. System 3 shows intensities that fell above 0.65 at the borders of the FOV and above 0.95 at the center of the FOV. A similar behavior was observed for system 1 but with lower intensity values at the left and right sides of the FOV. Finally, system 2 showed intensity values between 0.4 and 0.6 for $\sim 70\%$ of the FOV with values above the 0.6 at the left and right borders of it.

Fig. 8

Depiction of homogeneity test phantom used (a) with dimensions $\sim 18\times 18{\mathrm{cm}}^2$, hence larger than the evaluated FOVs. (b) Acquired images of the homogeneous material with systems 1 to 5. (c) Plot of normalized intensity across the $x$ pixel dimension of the sensor for each selected central region (shown in a) of the $y$ axis. Pixels containing company logos were set to 0 for display here.

3.4.

Test 5 - Spatial Distortion

To test spatial distortion from systems 1 to 5, the homogeneous fluorescent phantom used in uniformity testing is covered with a black plastic sheet with equidistant fluorescent points. A depiction of the phantom is shown in Fig. 9(a). This way fluorescence only leaked from the dotted areas. The acquired images per each system are shown in Fig. 9(b). Once images were acquired, distortion estimation was done based on the average distance between neighboring dots across the horizontal FOV. This process is described in Fig. S1(c) in the Supplementary Material for a single system. Two lines of dots located across the middle position of the FOV $y$-axis were selected. The intensities of dots contained across the $x$-axis were plotted, with an example shown in Fig. S1(c) in the Supplementary Material. Then the position of each peak was estimated based on the local maximum and peak values. The pixel distance from one peak to the other was then estimated by subtracting their position. These pixel distances between positions were then plotted as shown in Fig. 9(c) and were used to estimate the level of distortion within the evaluated systems.

Fig. 9

(a) Depiction of large area distortion phantom. (b) Images acquired by each system from the phantom with equidistant fluorescent points. The phantom was of dimensions $\sim 18\times 18{\mathrm{cm}}^2$, hence larger than the evaluated FOVs. (b) Acquired images with systems 1 to 5. (c) Plot of pixel distance between selected equidistant fluorescent points.

The resulting distances were normalized across the $y$-axis for visualization purposes, as shown in Fig. 9(c). In an ideal non-distorted image distances recorded between dots should be all equal to 1, which in the plot would be represented as a straight line across the selected horizontal dots. Results for system 3 show between dot distances above 0.9 for all dots, indicating that there was a lower level of distortion in the recorded image compared with other systems. This was also the case for system 1. System 2 displayed similar distance values above 0.9 for most dots, but values fell below this range on the right-most side of the FOV. System 5 shows distortion on the right side of the FOV as values fell between 0.85 and 0.9. Finally, system 4 displayed higher distortion in the left side of the FOV where values fell toward distances of 0.8.

3.5.

Test 6 - Optical Depth of Field

To calculate the DOF, images were acquired for each system with the parameters mentioned in Figs. S4 and S5 in the Supplementary Material. In this case, the sample was the resolution target used in Fig. 7(a), but only the $1.0\mathrm{lp}/\mathrm{mm}$ feature was of interest across the different positions, as depicted in Fig. 10. The systems were focused to their suggested WD, which corresponds to the values shown in Fig. 6. This plane of focus was defined as the distance 0. Then images of the resolution target were acquired at distances of 1 to 7 cm every 1 cm in both negative and positive directions. The negative values indicate distances further from the plane of focus but closer to the objective lens of each FGS system, and positive values indicate distances further away from the plane of focus and objective lens. Even though the evaluated systems had manual focusing arrangements for defined focal lengths, the sample was only focused when positioned at the 0-distance plane. This aims to illustrate the importance of considering the DOF in cases in which the surgeon focuses to a defined imaging $z$ plane but either the patient movement or movement of the imaging head can change the distance in a negative or positive $z$ position. This is of special importance for hand-held systems (e.g., systems 4 and 5). For those systems with magnification features (systems 3 and 5), magnification is not applied to cover the full FOV for all systems. CTF (%) values were calculated per each acquired image (at each $z$ distance position) in a similar process, as described in test 3 with Eq. (1), but only for the $1.0\mathrm{lp}/\mathrm{mm}$ element group 1. The CTF (%) values of both horizontal and vertical components were quantified separately, and the mean CTF (%) was calculated from them. Figure 10(a) displays example images acquired with each system at distances of $-5$, $-2$, 0, 2 and 6 cm, respectively. System 2 had a security feature indicating that the imaging head was too close to tissue; hence the considered negative distances go from 0 to $-6\mathrm{cm}$, and the positive distances go from 0 to 7 cm. The full set of images is shown in Fig. S3 in the Supplementary Material. Figure 10(c) shows the CTF (%) quantification results for all systems and mentioned $z$ positions.

Fig. 10

(a) Example of images acquired at $-5$, $-2$, 0 (plane of focus), 2 cm, and 6 cm. (b) Used elements for the quantification of the DOF metric. (c) CTF (%) as quantified for the selected $1.12\mathrm{lp}/\mathrm{mm}$ resolution for all of the evaluated distances departing from the plane of focus at the 0 position. Images are cropped to the target site for better visualization.

Systems 1 and 4 display higher CTF (%) for the $1.0\mathrm{lp}/\mathrm{mm}$ element within a range of $0\pm 2\mathrm{cm}$. System 3 shows the highest CTF (%) when the distance between the imaging head and the sample increases from 1 to 5 cm from the set distance. System 2 shows the largest CTF (%) range, covering values above 60% from $0\pm 5\mathrm{cm}$, whereas system 5 performs the best at $1\pm 5\mathrm{cm}$.

3.6.

Test 7 - Concentration Sensitivity

For sensitivity testing, a varied concentration target (QUEL Imaging, White River Jct, Vermont, United States) was used. The target mimics ICG fluorescence in the presence of tissue optical properties with (${\mu }_a$) of $\sim 0.010{\mathrm{mm}}^{-1}$ and reduced scattering coefficient (${\mu }_{s^{\prime }}$) of $\sim 1.9{\mathrm{mm}}^{-1}$ at 800 nm. The phantom contains nine wells of 1 cm diameter, with decreasing concentrations of 1000, 300, 100, 30, 10, 3, and 1 nm. It also contains a control well with 0 nm and a well with 800 nm emitting quantum dots (QD800), which can be excitable across wavelength bands 350 to 800 nm. The sample was measured with each system using the same parameters and modes reported in Figs. S4 and S5 in the Supplementary Material. The sample is depicted in Fig. 11(a).

Fig. 11

(a) Concentration variation imaging phantom with seven wells with different ICG nM concentrations, a 0 nm control well, and 800 nm emitting quantum dot well (QD). (b) Output images acquired from the phantom with each of the evaluated FGS systems. (c) Mean and standard deviation plot of each well per evaluated system. Calculated means and standard deviation per system were normalized to their maximum value for display purposes. The 800 nm quantum dots are a different probe than ICG with excitation from 350 to 800 nm; hence it is displayed separately.

The images of the concentration target, as acquired with each system, are shown in Fig. 11(b). In this case, the QD800 well is representative of the presence of room light sources that emit in the 350 to 633 nm range. Each system’s image is quantified by selecting well regions and calculating the mean intensity and standard deviation. Results for this quantification are plotted in Fig. 12(c) per each system. The mean and standard deviation values were normalized per system for display purposes. As shown in Fig. 12(c), eventhough systems 4 and 5 display higher sensitivity to the 1 nm concentration well, they display similar mean values for the 0 nm well. Systems 1, 2, and 3 display values approximating 0 for both 0 and 1 nm concentrations, which indicates a lower noise floor. In terms of saturation points, all systems display saturation at the 1000 and 300 nm wells. However for systems 2 and 3, the lowest concentrations of saturation were 100 and 30 nm, respectively. The importance of understanding the noise floor and saturation points is further discussed in the discussions and conclusions sections. The QD800 well that is excited from 350 to 633 nm showed higher intensity for system 5, indicating that, despite room lights being off, some light sources within these bands were present.

Fig. 12

(a) Depth imaging phantom with eight wells with ICG 1000 nm at different depths and a 0 nm control well with 6 mm of mimicking material. (b) Images acquired from the phantom with each of the evaluated FGS systems. (c) Mean and standard deviation plot of each well per evaluated system.

3.7.

Test 8 - Imaging Depth

To approximate how deep each FGS system can image through tissue, a similar phantom to the concentration one was used, but it has nine different 1 cm diameter wells. In this case, each well contains two layers: a top layer of non-fluorescent tissue mimicking material (non-fluorescent resin) with thickness variations from 0.5 to 6 mm, having homogeneous ${\mu }_a=0.019{\mathrm{mm}}^{-1}$ at 800 nm, followed by a bottom layer with 1000 nm ICG-equivalent material with ${\mu }_a=0.021{\mathrm{mm}}^{-1}$ and ${\mu }_{s^{\prime }}=0.27{\mathrm{mm}}^{-1}$ at 800 nm. One negative control well containing no ICG and only 6 mm of mimicking material was used. This phantom is pictured in Fig. 12(a).

Acquisition parameters per each system are consistent with the ones previously used. Images of the phantom as acquired with each system are shown in Fig. 12(b). For quantification, the intensities of each 1 cm well were quantified, and the mean and standard deviation were estimated. The resulting values per well/depth are plotted per FGS system in Fig. 12(c). The mean and standard deviation values were normalized to the maximum value per system for display purposes. For FGS systems, the control well, which contains 0 nm ICG and only mimicking material, display values higher than 0 intensity counts, with systems 1 and 3 reporting the lowest level of signal at the control well. As the depth of ICG decreases from 6 to 0.5 mm, the mean intensity and standard deviation increase to 1 for all systems. Even though systems 2, 4, and 5 display higher intensity at the control well, they also show higher intensity at depths from 1 to 6 mm, with system 2 showing more intensity at the 2 and 3 mm depths. Further hypothesizing of why the control well was not showing ideal values close to 0 for all systems is addressed in Sec. 4.

3.8.

Test 9 - Signal to Background Ratio

Even though test 7, in which concentration sensitivity was tested, gave an approximation of the noise floor of each FGS system, the images acquired from the depth phantom were also used for estimation of the signal to background ratio. For each image from an FGS system in Fig. 12(b), the wells containing ICG 1000 nm at depths of 2, 3, and 4 mm were used. A circular region with 0.5 units more than the well radius was selected for these 3 wells, as illustrated in Fig. 13(a). Then the well area was classified as the signal area and the surrounding regions as the background. Signal to background ratios (SBRs) of signal versus background are calculated by evaluating the ratio of the well intensity summed squared values over those of the background and displayed in dB units. Higher dB values are representative of lower background noise. Results for this test are displayed in Fig. 13(b) per each of the FGS systems.

Fig. 13

(a) Example of selected regions for signal (wells) and background (surrounding area). (b) Estimated signal to background ratios in dBs per each FGS system.

Systems 1, 2, and 3 show signal to background ratios greater than 20 dB with system 3 displaying the lowest level of background. Systems 4 and 5 display the highest levels of background with 10 and 13.1 dB, respectively. An important factor was the use of LUTs, which computationally associate input values to color corrected outputs. The range defined in each LUT is important as it can aid in defining parameters such as the system saturation and noise floor. Figure S2 in the Supplementary Material shows, for an example FGS system, how the use of different LUT tables, which are often denoted in FGS systems as different “modes,” can have an effect on the saturation points and noise floor of the image.