2.1.

Device Description

The layout of the optics and electronics inside Versicolour is shown in Fig. 1(a). A laptop controls the image acquisition sequence by triggering the LEDs in sequence, as shown in Fig. 1(b). To avoid spectral mixing, the fluorescence image from each channel is acquired with only one LED turned on at a time, with the camera triggered to acquire an image synchronously when the LED is turned on. Therefore, to acquire a three-color image, the trigger unit (NI9402, National Instruments, United States) was programmed to generate a set of three individual trigger pulses, where the pulse period dictated the exposure time (our preprogrammed exposure times were 15, 25, 50, 100, and 250 ms corresponding to 20, 10, 6, and 1.3 fps three-color imaging), see Fig. 1(b). All three channels had identical exposure times and gain due to camera firmware limitations. The pulsing architecture supersedes simultaneous two-color imaging⁷ by acquiring spectrally unconstrained acquisitions for each spectral channel. A previous technical configuration⁷ had been limited by the Bayer filter inherent in most color camera solutions, which preselects the spectral range for each channel and completely removes any NIR capability.

Fig. 1

(a) Electro-optical architecture of Versicolour and (b) triggering sequence for 25-ms exposure time for each individual channel resulting in 10 fps imaging speed for combined three-color images (minor delay between each pulse inserted for camera hardware to handle the image transfer).

The technical configuration of Versicolour was based on our previous work⁷ with significant improvements in adapting the architecture to three-color fluorescence endomicroscopy and molecular imaging applications. The improvements include the addition of a NIR imaging and a parallel spectroscopy channel (Flame-S spectrometer, Ocean Optics, United States). Three LEDs (470 nm LED M470L3, 625 nm LED M625L3, 780 nm LED M780F2, all from Thorlabs, United Kingdom) were combined using achromatic condensers for collimation (ACL2520U-A for 470 nm and 625 nm LEDs and ACL2520U-B for 780 nm LED) and dichroics for combination. A dichroic filter (FF506-DI03-25X36, Semrock, United States) combined collimated blue light from the 470-nm LED with red and NIR collimated beams, which were combined using a separate dichroic (FF652-DI01-25X36, Semrock). The custom multiband filterset was manufactured by Chroma, United States. The excitation and emission curves are shown in Figs. 2(a) and 2(b) along with absorption and emission curves from representative fluorescent dyes: nitrobenzoxadiazole (NBD), fluorescein, cyanine 5 (Cy5), methylene blue, and indocyanine green (ICG). Apart from methylene blue, we have tested all dyes listed. Current human in vivo imaging uses either approved (fluorescein and ICG) or nontoxic (methylene blue) dyes.¹⁹ For both NBD and Cy5, we are developing a range of clinically translatable SmartProbes.^6,20,21

Fig. 2

(a) Pass and stop bands of the custom three-color excitation filter along with the absorption curves of representative fluorescent dyes (NBD, fluorescein, Cy5, methylene blue, and ICG) are shown and (b) the pass and stop bands for the custom three-color emission filter and dichroic are shown along with the emission curves of representative fluorescent dyes for each channel (NBD, fluorescein, Cy5, methylene blue, and ICG).

A multiband excitation filter passes the collimated LED light to the imaging fiber bundle via a multiband dichroic and a microscope objective (MO) (RMS10X-PF, Olympus). The fluorescence generated at the distal end of the fiber was transmitted through the fiber and via a multiband emission filter and tube lens (ITL200, Thorlabs, United Kingdom) to a fast monochromatic CMOS camera (GS3-U3-23S6M-C, Grasshopper3, Point Grey Research, Canada). On the emission path, a 50:50 nonpolarizing beamsplitter (BS013, Thorlabs) transmits fluorescence into a spectrometer (Flame-S, Ocean Optics). The exposure time for spectrometer measurements was set to 2 s.

Custom real-time software was written in C++ with wxWidgets user interface classes. An intuitive GUI was developed for clinical use and the Versicolour unit was placed on a medically approved cart (see Fig. 3). To enable clinical translation, Versicolour was developed within the framework of a quality management system following the principles of BS EN 13485:2012. Risk management and mitigation were organized around BS EN 14971:2012. Software development followed the international standard BS EN 62304:2006 Medical Device Software principles. Versicolour passed electromagnetic compatibility (EMC) tests for radiated and conducted emissions (York EMC Services, Grangemouth, United Kingdom) and all locally required optical and electrical safety testing for use in the interventional pulmonology suite. Instrument enclosure mechanical designs (EG Technology Ltd., Cambridge) ensured ingress and electrical safety compliance following BS EN 60601-1.

Fig. 3

The medically approved cart with large display and the Versicolour unit can be easily transported between hospital departments.

The total cost of the bill of materials for one-off manufacture is $15000. The projected unit cost for 10 units (based on quotations) is <$5000 without spectrometer and <$8000 with spectrometer. As the design is breadboard based with relatively expensive electronics (e.g., National Instrument Cards), we believe that a commercial imaging design can be produced for less than $3000 for the manufacture of 10 units (without spectrometer). Furthermore, by deploying standard product design and optical engineering techniques (e.g., single metal block design for optics rather than breadboard based design), this can be reduced further. A similar single-color confocal scanning unit sells for 100,000$.²² One needs to take into account that regulatory aspects related to Food and Drug Administration approval in USA and Conformité Européene marking in the European Union can take up a significant fraction of the cost of the end unit. However, even in this respect we believe that the sheer simplicity of the design presented here warrants faster translation to clinic.

2.2.

Limit-of-Detection

The LoD for fluorescein (F6377, Sigma-Aldrich), Cy5 (777323, Sigma-Aldrich), and ICG (21980, Sigma Aldrich) solutions was determined using aqueous solutions of the fluorophores (5, 10, 20, and 30 nM in deionized water). The tip of the imaging fiber bundle was inserted into the solution, and several images were taken for each concentration with a 100-ms exposure time. A $100\times 100\text{pixel}$ area was selected from the central area of the imaging fiber bundle. Mean and standard deviation (SD) were calculated for the area and their ratio was ascribed as the signal-to-noise ratio (SNR). To remove variation attributable to the fiber bundle pattern, the SD value was derived from the SD of the same area but for a difference of two subsequent images. The difference removes the pattern while the noise inherent to image acquisition increases (by a factor of $\sqrt{2}$). To simplify matters, we defined the LoD as the detectable concentration of respective dyes resulting in a $2:1$ SNR.^7,23

2.3.

Chromatic Shift Test

To estimate the chromatic focal shift for all color channels (green, red, and NIR), the fiber imaging bundle was dipped into solutions of fluorescein, Cy5, and ICG (30 nM). A $Z$ axis stage (SM1Z, Thorlabs), which places the proximal side of the fiber imaging bundle into the focus, was moved along a $50\mu \mathrm{m}$ distance in steps of $1\mu \mathrm{m}$. A bandpass filter between 520 and 560 nm was placed in the fluorescence emission path for green channel measurements due to red fluorescence from the fiber at 470-nm excitation.^7,24 This allowed the fluorescence intensity to be evaluated from a single core versus focal position in front of the MO. Maximum fluorescence from the respective solutions indicated the optimal focus for a given channel.

A diluted solution of Constellation™ microspheres (ThermoFisher Scientific, United States) was loaded onto the distal tip of the imaging fiber and allowed to dry ($\sim 5\min$) and used to evaluate the image quality for fluorescence from microspheres in green, red, and NIR with the aim of establishing the image quality in the optimal focus.

2.4.

Ex vivo Human Lung Tissue Imaging with Labeled Cells and Bacteria

Ethics statement: All experiments using human samples in vitro were performed following approval of the appropriate regional ethics committee (REC), NHS Lothian (reference 15/ES/0094), and with informed consent of the patients.

Ex vivo human lung tissue, coincubated with mammalian and bacterial cells, was used as a model to evaluate cellular and bacterial imaging performance in multicolor scenarios. The $1\times 4\mathrm{mm}$ sections of excised human lung tissue were placed into a 96-well tissue culture plate (Corning) and incubated with $50\mu \mathrm{L}$ of prelabeled cells. The tissues were harvested from the periphery of specimens taken from the lung cancer resections. Overnight cultures of Staphylococcus aureus ATCC25923 grown in Luria-Bertani (LB) broth (37°C, 250 rpm) were adjusted to an optical density at 595 nm (${\mathrm{OD}}_{595}$) of 1 and washed in phosphate-buffered saline (PBS, Gibco). The bacteria were labeled with Cellvue^® Claret (Sigma-Aldrich), as per manufacturer’s instructions, and resuspended in a final volume of 1 mL PBS ($\sim 1\times {10}^9\mathrm{CFU}{\mathrm{mL}}^{-1}$). The murine macrophage cell-line RAW264.7 (ATCC) was grown in complete Dulbecco’s modified Eagle’s medium (Gibco) culture medium plus 1% pen/strep to 80% confluence in T75 tissue culture flasks. RAW264.7 cells were harvested and adjusted to $5\times {10}^5\text{cells}/\mathrm{mL}$. IR775 (IR-775 chloride, 544914, Sigma-Aldrich) was incubated with the cells at $10\mu \mathrm{M}$, 37°C, 30 min followed by washing three times in PBS by centrifugation (5 min, $400\times g$). Primary human neutrophils were isolated, as previously described,²⁵ with the number of retrieved neutrophils was determined with NucleoCounter NC-1000 (Chemo Metec). Neutrophils were resuspended at a concentration of $5\times {10}^5$ per mL in PBS and labeled with $1\mu \mathrm{M}$ calcein AM (Sigma-Aldrich), 30 min 37°C, followed by washing three times in PBS by centrifugation (5 min, $400\times g$). Following labeling, the cells were added to the ex vivo human lung tissue and imaged with Versicolour by placing the distal end of the imaging fiber (Alveoflex™, Mauna Kea Technologies, France) in direct contact with the tissue. Images were captured with a 25-ms exposure time.

2.5.

Sheep Lung Imaging

Ex vivo whole ventilated ovine lung experiments were performed to evaluate performance in a human size-relevant model and in particular to determine proof of concept for: (i) bacterial imaging with a gram-negative specific bacterial imaging SmartProbe²⁰ (three sheep lungs experiments) and (ii) blood flow imaging in NIR using ICG²⁶ (one sheep lung experiment).

Following the same methodology as described before Ref. 27, ovine lungs were obtained from ewes destined for cull and ventilated. Gram-negative Pseudomonas aeruginosa 3284 were cultured overnight in LB, adjusted to an ${\mathrm{OD}}_{595}$ 1 in PBS and labeled with Syto60 ($1\mu \mathrm{M}$, ThermoFisher Scientific), followed by washing three times in PBS providing a red counterstain. Delivery of bacteria and a gram-negative SmartProbe were performed as follows: bronchoscopy (Pentax EPM-1000) was used to guide to the distal lung location and a flexible APC probe (ERBE) catheter was then inserted into the working channel of the bronchoscope to instill 2 ml of bacteria ${\mathrm{OD}}_{595}$ 2 ($\sim {10}^9\mathrm{CFU}/\mathrm{ml}$). The ERBE catheter was removed and an Alveoflex™ imaging fiber bundle was passed through the working channel of the bronchoscope and guided to the subsegments, where bacteria had been instilled. Imaging was performed in the distal lung on at least two locations for at least 10 s for each imaging session. The imaging fiber was removed and 2 ml of the gram-negative SmartProbe ($80\mu \mathrm{g}/5\mathrm{ml}$) was delivered into the same location. The imaging fiber was then reinserted and guided to the subsegments with imaging was performed as previously. Between each delivery, the bronchoscope was cleaned and disinfected using 8% ${\mathrm{H}}_2{\mathrm{O}}_2$. Clinical Alveoflex™ (Mauna Kea Technologies) was used as the imaging fiber bundle with Versicolour. For ICG imaging, ovine lungs were obtained and perfused as above. The 2 ml of 1 mM ICG were added to 2 L of perfusate and imaging was performed 2 min after ICG was added to the perfusate. The exposure time of the camera was set to 50 ms.

2.6.

Human in vivo Imaging

Versicolour was deployed in a clinical study, clinicaltrials.gov identifier NCT02604862, following approval of the appropriate REC, NHS Lothian (reference 15/SS/0235), and with informed consent of the patients. In these cases, a clinical Alveoflex™ (Mauna Kea Technologies) was used as the imaging fiber bundle.

All patients were scheduled to undergo routine bronchoscopy in an outpatient procedure suite using local anesthetic and conscious sedation. Patients were recruited if they had an underlying diagnosis of diffuse parenchymal lung disease or suspected lung cancer. This allowed us to use our endomicroscopy system to characterize disease of both the airways and the alveolar space. As patients were already scheduled to undergo bronchoscopy, we were able to perform our imaging study in the same sitting as diagnostic bronchoscopic investigation, such as lavage, brushings, biopsy, and transbronchial ultrasound guided needle aspiration without complication.

Areas of interest were selected based on cross-sectional imaging (thoracic computerized tomography), which was used to determine which pulmonary segments were examined using endomicroscopy. For imaging of fibrotic and healthy alveolar regions, the bronchoscope was directed to the segment of interest before passing the Alveoflex™ down the working channel and past the point of vision. The endomicroscopy system enables real-time feedback as the fiber passes along bronchial structure before achieving a transbronchial push and emerging into the alveolar space with visual confirmation. For imaging of bronchial abnormalities such as lung cancer, the lesion was identified using the bronchoscope before the fiber was advanced to make contact under direct vision with the diseased mucosa. The total number of images acquired was 38,880 and the total number of spectra was 663. White light from the bronchoscope was briefly switched off during spectral acquisition for bronchus, but this was not required after transbronchial push into alveolar space. Bronchoscopy and endomicroscopy were completed in six patients (Table 1), and following the procedure, they were monitored for 4 h before discharge and all received a follow-up telephone call 24 h later. There were no serious adverse events recorded in this study.

Table 1

Six patients recruited to the clinical study. NSCLC, nonsmall cell lung cancer; SCLC, small cell lung cancer; IPF, idiopathic pulmonary fibrosis.

2.7.

Autofluorescence Background Suppression Methods

To reduce the effects of autofluorescence background from lung tissue, the following two methods were evaluated. First, in human lung imaging in situ, the method of Muldoon et al.¹³ was adopted, where the fiber was gently pulled back once the distal lung tissue was reached, with reduced pressure on the tissue reducing autofluorescence.

In addition, we further explored droplet capture onto the endoscope’s tip for tissue background removal. For droplet analysis, the tip of the fiber captures liquid from wet tissue through surface tension adsorption. We utilized the captured liquid for separate imaging or spectroscopy analysis, because the tissue background is inherently removed. This is especially useful for molecular imaging of enzyme activity probes, where the spectral signature may overlap with tissue autofluorescence if the enzyme reporter has a similar fluorescent reporter. To evaluate the methodology, we placed excised human lung tissue in two microwells of a 96-well plate. The first microwell contained $150\mu \mathrm{l}$ of PBS, whereas the second contained a $3\mu \mathrm{M}$ solution of fluorescein amidite (FAM) in $150\mu \mathrm{l}$ of PBS. Spectroscopy was performed on tissue by placing the fiber tip on tissue in the microwell. After this, the tip was retrieved and a spectrum was acquired of the droplet captured on the tip of the imaging fiber bundle. Retrieving the fiber tip from the solution captured $\sim 400\mathrm{nl}$ droplet by surface tension. Five spectra were taken from each tissue in the microwell and the respective droplets. Exposure time for each spectroscopy acquisition was 2 s.

2.8.

Imaging Settings

The imaging settings for Versicolour were as follows: images acquired were $960\times 600\text{pixels}$, camera exposure time was typically set to 25 ms (10 fps three-color imaging) unless otherwise stated. The exposure time needs to be adjusted so it acquires enough fluorescence from tissue and other optical signatures in all color channels. Our main clinical constraint has also been to achieve 10 fps in three-color imaging to allow instant viewing and reaction to endoscope positioning. After evaluation, we chose a 25-ms exposure time as it was adequate for most of our three-color imaging experiments. For each imaging experiment, a fluorescence background image was taken from a blackened Eppendorf filled with deionized water and subtracted from all acquired images. For displaying clinical images, we performed a division of acquired image with a selected lightfield image taken from the same sequence.

3.1.

Limit-of-Detection

LoD analysis is shown in Fig. 4(a) (LoD here defined as the concentration of fluorophore, which generates a SNR of $2:1$). The LoD was 5.7 nM for fluorescein, 2.4 nM for Cy5, and 8.4 nM for ICG for 100-ms exposure time (or 22.8 nM, 9.6 nM, and 33.6 nM, respectively, for 25-ms exposure time assuming read noise limited detection). As CMOS camera technologies are rapidly improving (as are increases in in LED optical power), reductions in camera noise will push LoDs to subnanomolar concentrations in dark background scenarios. In our previous implementation,⁷ LoDs for fluorescein and Cy5 were 10 and 50 nM, respectively, and thus, the LoD for fluorescein and Cy5 has improved by a factor of 2 and 20.8, respectively. The main reasons for these improvements are lower camera noise, improved collection efficiency due to monochrome detection, and increased LED optical power (4 mW on the output of the imaging fiber bundle for 470- and 625-nm LEDs and 1.9 mW for 780-nm LED). The effect is more dramatic in the red fluorescence channel, because previously in the color camera, the fiber fluorescence was detected in the red pixels of the Bayer filter, whereas now fiber fluorescence is detected during the 470-nm LED trigger (as shown before, the imaging fiber bundle has a long Stokes shift with 470-nm excitation^7,24). We subtract the background (fiber fluorescence and camera dark noise) for each color band, but as mentioned before,⁷ the Poisson noise of this background remains the limiting factor.

Fig. 4

(a) SNR for 5, 10, 20, and 30 nM solutions of fluorescein, Cy5, and ICG. (b) Contrast for each color channel depended on proximal fiber end location. Optimal foci for green and NIR channels are separated by $25\mu \mathrm{m}$.

3.2.

Chromatic Shift Evaluation

Figure 4(b) shows the separation of optimal foci for green, red, and NIR channels. Green, red, and NIR channels were equally separated by $13\mu \mathrm{m}$. Individual cores were visible across this range. Optimally, three-color imaging (green, red, and NIR) requires the focus to be set for the red channel [focus location at $25\mu \mathrm{m}$ in Fig. 4(b)]. For optimal two color imaging in green and red, the focus is set between green and red peaks [$17\mu \mathrm{m}$ in Fig. 4(b)].

To illustrate that the optimal focus for three-color imaging is sufficient to image small objects, we show in Fig. 5 an image of Constellation™ microspheres dried on the tip of the imaging fiber bundle. Figure 5(a) has the three channels displayed separately to illustrate overall color colocalization, and Fig. 5(b) has them merged in single RGB image. The two line profiles were chosen to show colocation of green and red fluorescence (line profile 1) and separately, red and NIR fluorescence (line profile 2), both in Fig. 5(c). There are no colocating three-color microspheres, and NIR microspheres were always much larger than a single core, thus limiting our analysis for this band. Green and red fluorescence from the small microspheres generated signals in a single core in line profile 1 while red and NIR fluorescence in line profile 2 matched each other well despite the NIR having lower SNR. For saturating regions in green, we observed core-to-core coupling,²⁸ which reduces contrast.

Fig. 5

(a) Constellation™ microspheres dried on the tip with each color channel displayed separately (green, red, and NIR) and merged in (b) with two line profiles shown in (c). Line profile 1 of a microsphere fluorescing shows green and red fluorescence. Line profile 2 of a larger microsphere shows red and NIR fluorescence.

3.3.

Multicolor ex vivo Human Lung Tissue Imaging

Figure 6 demonstrates three-color imaging in ex vivo human lung tissue at 10 fps. Macrophages fluorescing in the NIR (displayed as blue) and bacteria fluorescing in red can be seen in the same field-of-view (FoV) [Figs. 6(a)–6(c)]. Ex vivo human lung tissue autofluorescence is captured in the green fluorescence channel as well as calcein-labeled neutrophils [Fig. 6(c)]. Most of human lung tissue autofluorescence is derived from the elastin fiber network.² Figure 6(b) has tissue out of focus due to motion of the probe, whereas the lung elastin fiber network is sharper in Figs. 6(a) and 6(c). Figure 6(c) shows colocating macrophages, neutrophils, and bacteria toward the center of the image. While depth information is not available to guarantee colocation in 3-D, we were encouraged to pursue imaging scenarios such as this in vivo in situ and add technical methodologies to improve optical sectioning. In summary, we have demonstrated three-color cellular and bacterial resolution imaging across the visible to NIR spectrum.

Fig. 6

(a) and (b) Ex vivo human lung (green fluorescence) with macrophages [NIR shown as blue from IR775, white arrows pointing to three in (a)] and bacteria (red fluorescence from Cellvue^® Claret). Tissue in panel (b) is mostly out of focus (white arrows). (c) Ex vivo human lung (green fluorescence) with macrophages (NIR shown as blue from IR775), bacteria (red fluorescence from Cellvue^® Claret), and neutrophils (strong green fluorescence from calcein labelled neutrophil—arrow).

In future applications that potentially involve exogenously delivered fluorophore SmartProbes²⁰ to the distal lung, distinguishing signal from noise will be challenging if the fluorophores’ emission spectrum overlaps with elastin. Thus, background suppression techniques discussed below represent an important aspect for in vivo imaging.

3.4.

Ventilated Ovine Lung Imaging

Moving from ex vivo imaging of small sections of patient-derived human lung tissue, the next stage of development was to demonstrate utility to derive optical signatures in a human size relevant model. In this regard, we pursued initial proof of concept in a ventilated and perfused ovine lung model in a dedicated setup [Fig. 7(a)]. The imaging bundle was navigated to the distal alveolar regions where imaging of bacteria is present [the bacteria were counterstained with Syto60 (red) prior to instilling into the lung and targeted with bacteria-specific SmartProbe (green) in situ]. Figure 7(b) shows example frames from one ex vivo ventilated sheep lung, each with representative green and corresponding red channels imaged simultaneously. No green punctate signals were observable in the green channel images prior to instilling gram-negative SmartProbe while red punctate Syto60 labeled bacteria were visible [top row in Fig. 7(b)]. After instilling gram-negative SmartProbe (with NBD green fluorophore), colocating red and green punctate signals were observable [bottom row in Fig. 7(b)]. The same results were obtained in the further two ovine lungs tested.

Fig. 7

(a) Perfusion set-up used in sheep lung experiments containing the following components: 1, organ chamber; 2, suction reservoir; 3, perfusate reservoir; 4, heater; 5, pump control; 6, gas membrane diffuser; 7, centrifugal pump head. (b) Ex vivo ventilated sheep lungs (each row) with green channel, corresponding red channel and combined image (three columns) prior to instilling gram-negative SmartProbe onto gram-negative bacteria (first row) and after instilling gram-negative SmartProbe onto gram-negative bacteria (second row). (c) Incoming wave of NIR fluorescence from ICG in the perfusate filling the distal lung of the ventilated and perfused ovine lungs.

Figure 7(c) shows sheep lung imaging snapshots with perfusate inflow. The ICG stained perfusate flow was observable in the distal lung for each pump cycle of the perfusate entering the FoV. The imaging fiber bundle was embedded in the distal lung to keep the FoV static. The NIR signal from ICG was diffused and blood vessels were not observable, probably due to trauma caused by the fiber insertion. After 20 min, the ICG started to stain the tissue.

3.5.

Clinical Imaging in the Distal Human Lung

We embarked upon an exploratory clinical study with Versicolour with the primary aim to generate proof of concept data that Versicolour could delineate autofluorescent signatures of pulmonary elastin organization. We thus selected patients in whom the elastin network would be expected to be disordered. We used CT scans to help guide imaging to normal areas as control images.

Figure 8 demonstrates three in vivo imaging scenarios of normal bronchial epithelium (left column). Two example images with microvessels are shown in Fig. 8 (middle column). The alveolar space was reached (Fig. 8, right column) after pushing the imaging fiber bundle gently into acinar subunits. Alveolar septa are clearly delineated in normal alveolar space images (Fig. 8, right column). Broken epithelium in nonsmall cell lung cancer (NSCLC) is shown in Fig. 9, left column, whereas broken epithelium in small cell lung cancer (SCLC) is shown in Fig. 9, middle column. Fibrotic lung (alveolar space) is shown in Fig. 9, right column. Overall, the level of autofluorescence varied from patient to patient but also within the same patient. Some structures fluoresce strongly (Fig. 9, right column, fibrotic lung) while bronchial epithelium and some normal alveolar structures fluoresce weakly resulting in low SNR images (Fig. 8, top right and Fig. 9, middle column lower image). The varying autofluorescence is likely to be attributable to varying abundance of elastin and collagen in the native structures. Our comparison between normal and disrupted tissue was not blind in this clinical study. Clinically, prior information from CT and white-light bronchoscopy provides important input in judging endomicroscopy imaging.

Fig. 8

Human in vivo imaging with examples of bronchial epithelium (left column, arrow pointing in the direction of normal elastic fibred network), distal lung with blood microvessel (middle column, arrow pointing in the direction of the vessel), and normal alveolar space lung (right column, arrow pointing to alveolar septa). All images were scaled individually for intensity to maximize contrast.

Fig. 9

The disturbed bronchial epithelium in NSCLC (left column, arrow pointing at fibred networks crossing each other) and SCLC (middle column, arrow pointing at fibred networks crossing each other). Fibrotic (distal) lung with disturbed alveolar structure is shown in right column, no alveolar septa can be found.

Given the widefield capture of images and the lack of sectioning, background autofluorescence contributes to “noisy” images. In our clinical experience in the distal lung (alveolar regions), this can be significantly attenuated by a small retraction ($\sim 200$ to $400\mu \mathrm{m}$) of the imaging bundle (see Fig. 10).

Fig. 10

Background suppression by retracting the fiber. (a) and (b) High background image in alveolar space can be improved if the clinician retracts the fiber in order to reduce the amount of elastin pressed beneath the imaging fiber bundle.

In addition, we explored background suppression by capturing droplets on the endoscope tip ex vivo. Figure 11 illustrates how the droplet suppresses tissue autofluorescence background. Spectra on tissue without FAM [Fig. 11(a)] and with FAM [Fig. 11(b)] are similar in intensity and shape, making it difficult to detect the presence of FAM. Droplet from the tissue without FAM does not fluoresce [Fig. 11(c)], whereas the droplet from tissue with FAM demonstrates strong spectra [Fig. 11(d)]. Droplet spectroscopy or imaging therefore allows for background suppression and could be useful for accurate enzyme activity analysis, where droplet capture is possible in vivo.

Fig. 11

While ex vivo human lung tissue (a) without FAM and (b) with FAM has similar spectra, (c) the droplet from tissue well without FAM does not fluoresce (d) while the droplet from tissue well with FAM has spectra corresponding to FAM.

3.6.

Discussion

As the only currently clinically approved endomicroscopy system for respiratory medicine is probe-based confocal laser endomicroscopy, clinicians rely on autofluorescence³ or intravenous fluorescein for alveolar imaging²⁹ and microvessel localization.³⁰ Other studies have used acriflavin hydrochloride topical staining for cellular imaging in vivo³¹ in green. Single color systems suffer from overlaps between green autofluorescence and green fluorescence staining, so there is an ongoing effort to verify red-shifted fluorescence cellular imaging in vivo such as with methylene blue.⁴ Our platform supports simultaneous imaging for numerous scenarios, such as methylene blue for cellular imaging in red and ICG for microvessel localization in NIR. We also observed weak red autofluorescence from alveoli during in vivo imaging in two out of six patients, but it is difficult to draw conclusions from such a small sample (see Fig. 12 for weak red fluorescence from alveolar septa).

Fig. 12

In two patients, alveolar space was visible in green and red channels. Green channel and matching red channel are combined into a single two color image. (a) Green channel, (b) red channel, and (c) combined.

With the development of optical imaging SmartProbes that are likely to be spread across the optical spectrum, the utility of modular cost effective solutions will be essential to realize the potential of OMI. In the case of Versicolour, the LEDs and filters can be simply interchanged to accommodate spectral windows.

The droplet method explored is very practical for single pass capture of surface liquids. While it exploits naturally occurring biofouling, it is also limited to single capture in vivo. In addition, it is not practical for the transbronchial pass. Despite the limitations, we feel it to be a very practical method for background suppression in all other situations. While we tested the method with spectroscopy, it is also applicable to imaging. For example, for imaging bacteria labeled in green where tissue background fluorescence masks most of them, we have observed more bacterial “dots” to reappear in the droplet on dark background thus improving detection.

In widefield imaging scenarios, optical sectioning remains an issue. Fluorescence emanates from up to $250\mu \mathrm{m}$ away and reaches the camera through the imaging fiber bundle.²¹ While this reduces contrast by increasing background, the system performs well for image guidance into distal lung tissue and bacterial imaging despite the fact that we do not perform any image processing. In addition, colocalization of objects in three channels will be a challenge in fast moving environments during in vivo imaging. We have found that a 25-ms exposure time is a good compromise in this respect. We have also found that exposure times above 100 ms cause significant blur during in vivo imaging. Our ambition in future work will be to address both 2-D and 3-D colocalization in order to bring endomicroscopy closer to the performance of bench top fluorescence microscopy.

Further technical developments include adding image processing,³² adopting structured illumination techniques,^33–35 and exploiting the acquisition speed of low noise CMOS cameras. These modular improvements offer a broad potential platform to exploit fluorescence-based endomicroscopy coupled with the emergence of low-cost disposable imaging fibers.³⁶