SignificanceOral cancer surgery requires accurate margin delineation to balance complete resection with post-operative functionality. Current in vivo fluorescence imaging systems provide two-dimensional margin assessment yet fail to quantify tumor depth prior to resection. Harnessing structured light in combination with deep learning (DL) may provide near real-time three-dimensional margin detection.AimA DL-enabled fluorescence spatial frequency domain imaging (SFDI) system trained with in silico tumor models was developed to quantify the depth of oral tumors.ApproachA convolutional neural network was designed to produce tumor depth and concentration maps from SFDI images. Three in silico representations of oral cancer lesions were developed to train the DL architecture: cylinders, spherical harmonics, and composite spherical harmonics (CSHs). Each model was validated with in silico SFDI images of patient-derived tongue tumors, and the CSH model was further validated with optical phantoms.ResultsThe performance of the CSH model was superior when presented with patient-derived tumors (P-value<0.05). The CSH model could predict depth and concentration within 0.4 mm and 0.4 μg/mL, respectively, for in silico tumors with depths less than 10 mm.ConclusionsA DL-enabled SFDI system trained with in silico CSH demonstrates promise in defining the deep margins of oral tumors.
Fluorescence-guided surgery systems employed during oral cancer resection help detect the lateral margin yet fail to quantify the deep margins of the tumor prior to resection. Without comprehensive quantification of three-dimensional tumor margins, complete resection remains challenging. While interoperative techniques to assess the deep margin exist, they are limited in precision, leaving an unmet need for a system that can quantify depth. Our group is developing a deep learning (DL)-enabled fluorescence spatial frequency domain imaging (SFDI) system to address this limitation. The SFDI system captures fluorescence (F) and reflectance (R) images that contain information on tissue optical properties (OP) and depth sensitivity across spatial frequencies. Coupling DL with SFDI imaging allows for the near-real time construction of depth and concentration maps. Here, we compare three DL architectures that use SFDI images as inputs: i) F+OP, where OP (absorption and scattering) are obtained analytically from reflectance images; ii) F+R; iii) F/R. Training the three models required 10,000 tumor samples; synthetic tumors derived from composite spherical harmonics circumvented the need for patient data. The synthetic tumors were passed to a diffusion-theory light propagation model to generate a dataset of artificial SFDI images for DL training. Two oral cancer models derived from MRI of patient tongue tumors are used to evaluate DL performance in: i) in silico SFDI images ii) optical phantoms. These studies evaluate how system performance is affected by the SFDI input data and DL architectures. Future studies are required to assess system performance in vivo.
KEYWORDS: Fluorescence tomography, Cancer, Surgery, Luminescence, Spatial frequencies, Monte Carlo methods, Imaging systems, Animal model studies, Tumors, Data modeling
Fluorescence imaging during to oral cancer surgery is typically 2D, yielding limited information on tumor depth. Here, we continue the development of a spatial frequency domain imaging (SFDI) system for 3D fluorescence imaging. A deep convolutional neural network takes as inputs SFDI-computed absorption, scattering and spatial-frequency fluorescence images, and yields images of fluorescence concentration and tumour depth. The model is trained using in silico data from Monte Carlo simulations of geometric tumor shapes (e.g., cylinder, spherical harmonics). Initial results yield average depth errors of <0.1 mm. Experiments are conducted in agar phantoms based on patient imaging.
Accelerating innovation in the space of fluorescence imaging for surgical applications has increased interest in safely and expediently advancing these technologies to clinic through Food and Drug Administration- (FDA-) compliant trials. Conventional metrics for early phase trials include drug safety, tolerability, dosing, and pharmacokinetics. Most procedural imaging technologies rely on administration of an exogenous fluorophore and concurrent use of an imaging system; both of which must receive FDA approval to proceed to clinic. Because fluorophores are classified as medical imaging agents, criteria for establishing dose are different, and arguably more complicated, than therapeutic drugs. Since no therapeutic effect is desired, medical imaging agents are ideally administered at the lowest dose that achieves adequate target differentiation. Because procedural imaging modalities are intended to enhance and/or ease proceduralists’ identification or assessment of tissues, beneficial effects of these technologies may manifest in the form of qualitative endpoints such as: 1) confidence; 2) decision-making; and 3) satisfaction with the specified procedure. Due to the rapid expansion of medical imaging technologies, we believe that our field requires standardized criteria to evaluate existing and emerging technologies objectively so that both quantitative and qualitative aspects of their use may be measured and useful comparisons to assess their relative value may occur. Here, we present a 15-item consensus-based survey instrument to assess the utility of novel imaging technologies from the proceduralist’s standpoint.
Clinical trials with novel fluorescence contrast agents for head and neck cancer are driving new applications for fluorescence-guided surgery. Two-dimensional fluorescence imaging systems, however, provide limited in vivo assessment capabilities to determine tumor invasion depth below the mucosal surface. Here, we investigate the use of spatial frequency domain imaging (SFDI) methods for sub-surface fluorescence in tissue-simulating oral cancer phantoms. A two-step profile-correction approach for SFDI is under development to account for the complex surface topography of the oral cavity. First, for structured-illumination estimation of the surface profile, we are evaluating gray code and phase shift profilometry methods in agar-based oral cavity phantoms to maximize resolution and minimize sensitivity to surface discontinuities. Second, for profile-correction of the diffuse reflectance, global lighting effects within the oral cavity – analogous to an integrating sphere – are modeled using a multi-bounce numerical model. Subsurface fluorescence imaging is enabled based on the variations in optical sampling depth that result from changes in spatial frequency. An analytical depth recovery approach is based on a numerical diffusion theory model for semi-infinite fluorescence slabs of variable thickness. Depth estimation is evaluated in an agar-based phantom with fluorescence inclusions of thicknesses 1-5.5 mm originating from the top surface (“iceberg model”). Future clinical studies are necessary to assess in vivo performance and intraoperative workflow.
Current intraoperative methods to assess tumor invasion depth in mucosal oral cancer provide limited real-time information. The advent of targeted fluorescence contrast agents for head and neck cancer is a promising innovation, but surgical imaging systems typically provide only two-dimensional views. Here, we investigate the use of an image-guided fluorescence tomography (igFT) system to estimate the depth of tumor invasion in tissue-simulating oral cancer phantoms. Implementation of non-contact diffuse optical tomography using finite-element software (NIRFAST) is enabled with geometric data from intraoperative cone-beam CT (CBCT) imaging and surgical navigation. The tissue phantoms used gelatin for the background (5% for fat, 10% for muscle) and 2% agar for palpable, tumor-like inclusions. Standard agents were used for absorption (hemoglobin), scattering (Intralipid), fluorescence (indocyanine green), and CT contrast (iohexol). The agar inclusions were formed using 3D printed molds, and positioned at the surface of the gelatin background to mimic mucosal tumor invasion (an “iceberg” model). Simulations and phantom experiments characterize fluorescence tomography performance across a range of tumor invasion depths. To aid surgical visualization, the fluorescence volume is converted to a colored surface indicating tumor depth, and overlaid on the navigated endoscopic video. Clinical studies are necessary to assess in vivo performance and intraoperative workflow.
Intraoperative visualization of molecular processes delineated by fluorescence contrast agents offers the potential for increased surgical precision and better patient outcomes. To exploit fully the clinical potential for targeted fluorescence guidance, there is a growing need to develop high-resolution, quantitative imaging systems suitable for surgical use. Diffuse optical fluorescence tomography (DOFT) systems in pre-clinical and diagnostic imaging applications have demonstrated improvements in fluorescence quantification with the addition of a priori data from structural imaging modalities (e.g., MR, CT). Here, we investigate the use of a cone-beam CT (CBCT) surgical guidance system to generate spatial priors for intraoperative DOFT. Imaging and localization data is incorporated directly into a finite element method DOFT implementation (NIRFAST) at multiple stages. First, CBCT data from an intraoperative flat-panel C-arm is used to generate tetrahedral meshes. Second, optical tracking of laser and camera devices enables an adaptable non-contact DOFT approach to accommodate various anatomical sites and acquisition geometries. Finally, anatomical segmentations from CBCT are included in the optical reconstruction process using Laplacian-type regularization (“soft spatial priors”). Calibration results showed that light rays between the tissue surface and navigated optical devices were mapped with sub-millimeter accuracy. Liquid phantom experiments determined the improvements in quantification of fluorescence yield, with errors of 85% and <20% for no priors and spatial priors, respectively. CBCT-DOFT fusion in a VX2-tumor rabbit model delineated contrast enhancement using a dual CT/optical liposomal nanoparticle. These developments motivate future translation and evaluation in an ongoing CBCT-guided head and neck surgery patient study.
A freehand, non-contact diffuse optical tomography (DOT) system has been developed for multimodal imaging with
intraoperative cone-beam CT (CBCT) during minimally-invasive cancer surgery. The DOT system is configured for
near-infrared fluorescence imaging with indocyanine green (ICG) using a collimated 780 nm laser diode and a nearinfrared
CCD camera (PCO Pixelfly USB). Depending on the intended surgical application, the camera is coupled to
either a rigid 10 mm diameter endoscope (Karl Storz) or a 25 mm focal length lens (Edmund Optics). A prototype flatpanel
CBCT C-Arm (Siemens Healthcare) acquires low-dose 3D images with sub-mm spatial resolution. A 3D mesh is
extracted from CBCT for finite-element DOT implementation in NIRFAST (Dartmouth College), with the capability for
soft/hard imaging priors (e.g., segmented lymph nodes). A stereoscopic optical camera (NDI Polaris) provides real-time
6D localization of reflective spheres mounted to the laser and camera. Camera calibration combined with tracking data is
used to estimate intrinsic (focal length, principal point, non-linear distortion) and extrinsic (translation, rotation) lens
parameters. Source/detector boundary data is computed from the tracked laser/camera positions using radiometry
models. Target registration errors (TRE) between real and projected boundary points are ~1-2 mm for typical acquisition
geometries. Pre-clinical studies using tissue phantoms are presented to characterize 3D imaging performance. This
translational research system is under investigation for clinical applications in head-and-neck surgery including oral
cavity tumour resection, lymph node mapping, and free-flap perforator assessment.
A prototype mobile C-arm for cone-beam CT (CBCT) has been translated to a prospective clinical trial in head and neck
surgery. The flat-panel CBCT C-arm was developed in collaboration with Siemens Healthcare, and demonstrates both
sub-mm spatial resolution and soft-tissue visibility at low radiation dose (e.g., <1/5th of a typical diagnostic head CT).
CBCT images are available ~15 seconds after scan completion (~1 min acquisition) and reviewed at bedside using
custom 3D visualization software based on the open-source Image-Guided Surgery Toolkit (IGSTK). The CBCT C-arm
has been successfully deployed in 15 head and neck cases and streamlined into the surgical environment using human
factors engineering methods and expert feedback from surgeons, nurses, and anesthetists. Intraoperative imaging is
implemented in a manner that maintains operating field sterility, reduces image artifacts (e.g., carbon fiber OR table) and
minimizes radiation exposure. Image reviews conducted with surgical staff indicate bony detail and soft-tissue
visualization sufficient for intraoperative guidance, with additional artifact management (e.g., metal, scatter) promising
further improvements. Clinical trial deployment suggests a role for intraoperative CBCT in guiding complex head and
neck surgical tasks, including planning mandible and maxilla resection margins, guiding subcranial and endonasal
approaches to skull base tumours, and verifying maxillofacial reconstruction alignment. Ongoing translational research
into complimentary image-guidance subsystems include novel methods for real-time tool tracking, fusion of endoscopic
video and CBCT, and deformable registration of preoperative volumes and planning contours with intraoperative CBCT.
Methods for accurate registration and fusion of intraoperative cone-beam CT (CBCT) with endoscopic video have been
developed and integrated into a system for surgical guidance that accounts for intraoperative anatomical deformation and
tissue excision. The system is based on a prototype mobile C-Arm for intraoperative CBCT that provides low-dose 3D
image updates on demand with sub-mm spatial resolution and soft-tissue visibility, and also incorporates subsystems for
real-time tracking and navigation, video endoscopy, deformable image registration of preoperative images and surgical
plans, and 3D visualization software. The position and pose of the endoscope are geometrically registered to 3D CBCT
images by way of real-time optical tracking (NDI Polaris) for rigid endoscopes (e.g., head and neck surgery), and
electromagnetic tracking (NDI Aurora) for flexible endoscopes (e.g., bronchoscopes, colonoscopes). The intrinsic (focal
length, principal point, non-linear distortion) and extrinsic (translation, rotation) parameters of the endoscopic camera
are calibrated from images of a planar calibration checkerboard (2.5×2.5 mm2 squares) obtained at different
perspectives. Video-CBCT registration enables a variety of 3D visualization options (e.g., oblique CBCT slices at the
endoscope tip, augmentation of video with CBCT images and planning data, virtual reality representations of CBCT
[surface renderings]), which can reveal anatomical structures not directly visible in the endoscopic view - e.g., critical
structures obscured by blood or behind the visible anatomical surface. Video-CBCT fusion is evaluated in pre-clinical
sinus and skull base surgical experiments, and is currently being incorporated into an ongoing prospective clinical trial in
CBCT-guided head and neck surgery.
esthetic appearance is one of the most important factors for reconstructive surgery. The current practice of maxillary
reconstruction chooses radial forearm, fibula or iliac rest osteocutaneous to recreate three-dimensional complex
structures of the palate and maxilla. However, these bone flaps lack shape similarity to the palate and result in a less
satisfactory esthetic. Considering similarity factors and vasculature advantages, reconstructive surgeons recently
explored the use of scapular tip myo-osseous free flaps to restore the excised site. We have developed a new method that
quantitatively evaluates the morphological similarity of the scapula tip bone and palate based on a diagnostic volumetric
computed tomography (CT) image. This quantitative result was further interpreted as a color map that rendered on the
surface of a three-dimensional computer model. For surgical planning, this color interpretation could potentially assist
the surgeon to maximize the orientation of the bone flaps for best fit of the reconstruction site. With approval from the
Research Ethics Board (REB) of the University Health Network, we conducted a retrospective analysis with CT image
obtained from 10 patients. Each patient had a CT scans including the maxilla and chest on the same day. Based on this
image set, we simulated total, subtotal and hemi palate reconstruction. The procedure of simulation included volume
segmentation, conversing the segmented volume to a stereo lithography (STL) model, manual registration, computation
of minimum geometric distances and curvature between STL model. Across the 10 patients data, we found the overall
root-mean-square (RMS) conformance was 3.71± 0.16 mm
High-quality intraoperative 3D imaging systems such as cone-beam CT (CBCT) hold considerable promise for imageguided
surgical procedures in the head and neck. With a large amount of preoperative imaging and planning information
available in addition to the intraoperative images, it becomes desirable to be able to integrate all sources of imaging
information within the same anatomical frame of reference using deformable image registration. Fast intensity-based
algorithms are available which can perform deformable image registration within a period of time short enough for
intraoperative use. However, CBCT images often contain voxel intensity inaccuracy which can hinder registration
accuracy - for example, due to x-ray scatter, truncation, and/or erroneous scaling normalization within the 3D
reconstruction algorithm. In this work, we present a method of integrating an iterative intensity matching step within the
operation of a multi-scale Demons registration algorithm. Registration accuracy was evaluated in a cadaver model and
showed that a conventional Demons implementation (with either no intensity match or a single histogram match)
introduced anatomical distortion and degradation in target registration error (TRE). The iterative intensity matching
procedure, on the other hand, provided robust registration across a broad range of intensity inaccuracies.
A system for intraoperative cone-beam CT (CBCT) surgical guidance is under development and translation to trials in
head and neck surgery. The system provides 3D image updates on demand with sub-millimeter spatial resolution and
soft-tissue visibility at low radiation dose, thus overcoming conventional limitations associated with preoperative
imaging alone. A prototype mobile C-arm provides the imaging platform, which has been integrated with several novel
subsystems for streamlined implementation in the OR, including: real-time tracking of surgical instruments and
endoscopy (with automatic registration of image and world reference frames); fast 3D deformable image registration (a
newly developed multi-scale Demons algorithm); 3D planning and definition of target and normal structures; and
registration / visualization of intraoperative CBCT with the surgical plan, preoperative images, and endoscopic video.
Quantitative evaluation of surgical performance demonstrates a significant advantage in achieving complete tumor
excision in challenging sinus and skull base ablation tasks. The ability to visualize the surgical plan in the context of
intraoperative image data delineating residual tumor and neighboring critical structures presents a significant advantage
to surgical performance and evaluation of the surgical product. The system has been translated to a prospective trial
involving 12 patients undergoing head and neck surgery - the first implementation of the research prototype in the
clinical setting. The trial demonstrates the value of high-performance intraoperative 3D imaging and provides a valuable
basis for human factors analysis and workflow studies that will greatly augment streamlined implementation of such
systems in complex OR environments.
Surgical simulation has become a critical component of surgical practice and training in the era of high-precision image-guided
surgery. While the ability to simulate surgery of the paranasal sinuses and skull base has been conventionally
limited to 3D digital simulation or cadaveric dissection, we have developed novel methods employing rapid prototyping
technology and 3D printing to create high-fidelity models from real patient images (CT or MR). Such advances allow
creation of patient-specific models for preparation, simulation, and training before embarking on the actual surgery. A
major challenge included the development of novel material formulations compatible with the rapid prototyping process
while presenting anatomically realistic flexibility, cut-ability, drilling purchase, and density (CT number). Initial studies
have yielded realistic models of the paranasal sinuses and skull base for simulation and training in image-guided surgery.
The process of model development and material selection is reviewed along with the application of the phantoms in
studies of high-precision surgery guided by C-arm cone-beam CT (CBCT). Surgical performance is quantitatively
evaluated under CBCT guidance, with the high-fidelity phantoms providing an excellent test-bed for reproducible
studies across a broad spectrum of challenging surgical tasks. Future work will broaden the atlas of models to include
normal anatomical variations as well as a broad spectrum of benign and malignant disease. The role of high-fidelity
models produced by rapid prototyping is discussed in the context of patient-specific case simulation, novel technology
development (specifically CBCT guidance), and training of future generations of sinus and skull base surgeons.
High-performance intraoperative imaging is essential to an ever-expanding scope of therapeutic procedures ranging from
tumor surgery to interventional radiology. The need for precise visualization of bony and soft-tissue structures with
minimal obstruction to the therapy setup presents challenges and opportunities in the development of novel imaging
technologies specifically for image-guided procedures. Over the past ~5 years, a mobile C-arm has been modified in
collaboration with Siemens Medical Solutions for 3D imaging. Based upon a Siemens PowerMobil, the device includes:
a flat-panel detector (Varian PaxScan 4030CB); a motorized orbit; a system for geometric calibration; integration with
real-time tracking and navigation (NDI Polaris); and a computer control system for multi-mode fluoroscopy,
tomosynthesis, and cone-beam CT. Investigation of 3D imaging performance (noise-equivalent quanta), image quality
(human observer studies), and image artifacts (scatter, truncation, and cone-beam artifacts) has driven the development
of imaging techniques appropriate to a host of image-guided interventions. Multi-mode functionality presents a valuable
spectrum of acquisition techniques: i.) fluoroscopy for real-time 2D guidance; ii.) limited-angle tomosynthesis for fast
3D imaging (e.g., ~10 sec acquisition of coronal slices containing the surgical target); and iii.) fully 3D cone-beam CT
(e.g., ~30-60 sec acquisition providing bony and soft-tissue visualization across the field of view). Phantom and cadaver
studies clearly indicate the potential for improved surgical performance - up to a factor of 2 increase in challenging
surgical target excisions. The C-arm system is currently being deployed in patient protocols ranging from brachytherapy
to chest, breast, spine, and head and neck surgery.
The application of high-performance flat-panel detectors (FPDs) to dual-energy (DE) imaging offers the potential for dramatically improved detection and characterization of subtle lesions through reduction of "anatomical noise," with applications ranging from thoracic imaging to image-guided interventions. In this work, we investigate DE imaging performance from first principles of image science to preclinical implementation, including: 1.) generalized task-based formulation of NEQ and detectability as a guide to system optimization; 2.) measurements of imaging performance on a DE imaging benchtop; and 3.) a preclinical system developed in our laboratory for cardiac-gated DE chest imaging in a research cohort of 160 patients. Theoretical and benchtop studies directly guide clinical implementation, including the advantages of double-shot versus single-shot DE imaging, the value of differential added filtration between low- and high-kVp projections, and optimal selection of kVp pairs, filtration, and dose allocation. Evaluation of task-based NEQ indicates that the detectability of subtle lung nodules in double-shot DE imaging can exceed that of single-shot DE
imaging by a factor of 4 or greater. Filter materials are investigated that not only harden the high-kVp beam (e.g., Cu or
Ag) but also soften the low-kVp beam (e.g., Ce or Gd), leading to significantly increased contrast in DE images. A preclinical imaging system suitable for human studies has been constructed based upon insights gained from these theoretical and experimental studies. An important component of the system is a simple and robust means of cardiac-gated DE image acquisition, implemented here using a fingertip pulse oximeter. Timing schemes that provide cardiac-gated
image acquisition on the same or successive heartbeats is described. Preclinical DE images to be acquired under research protocol will afford valuable testing of optimal deployment, facilitate the development of DE CAD, and support comparison of DE diagnostic imaging performance to low-dose CT and radiography.
Future planetary exploration missions will aim at landing a spacecraft in hazardous regions of a planet, thereby requiring an ability to autonomously avoid surface obstacles and land at a safe site. Landing safety is defined in terms of the local topography-slope relative to gravity and surface roughness-and landing dynamics, requireing an impact velocity lower than a given tolerance. In order to meet these challenges, a LIDAR-based Autonomous Planetary landing System (LAPS) was developed, combining the three-dimensional cartographic capabilities of the LIDAR with autonomous 'intelligent' software for interpreting the data and for guiding the Lander to the safe site. This paper provides an overview of the LAPS ability to detect obstacles, identify a safe site and support the navigation of the Lander to the detected safe site. It also demonstrates the performance of the system using real LIDAR data taken over a physical emulation of a Mars terrain.
The two Viking missions of the 1970's are a testimony to the success of our technological capability when it is driven by consuming curiosity and sense of adventure. In the case of Viking, the national spirit in the United States supported an assemblage of equally spirited expertise within NASA to determine if life existed on Mars, and within the defined science of those missions to establish if conditions on Mars might support life. The technological successes of Viking led to a confusion of interpretations for the issue of life on Mars. This confusion in turn led to polarities in the scientific community and a subsequent resting period of some years for the enthusiasm required to support continued investigation of the potential existence of life on Mars, and indeed elsewhere in our solar system.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.