Epilepsy, a common neurological disorder causing recurring seizures. Magnetic Resonance-guided Laser Interstitial Thermal Therapy (MRgLITT) is a promising minimally-invasive technique to ablate the target especially for drug resistance epilepsy. MRgLITT employs a laser fiber to ablate brain tissue through heat deposition, offering real-time monitoring through Magnetic Resonance (MR) thermometry images and precise treatment planning using MRI planning images. In this study, we developed an AI-based approach utilizing a U-Net model, a convolutional neural network architecture widely used for image to image translation, to predict MR thermometry images from anatomical MRI planning images from a dataset of 81 patients with mesial temporal lobe epilepsy. The model’s performance was evaluated on a test dataset using the structural similarity index (SSIM) and root mean squared error (RMSE).
It is common for facial reconstructive surgeons to take pre- and post-operative images from patients to keep track of their healing progress. However, current guidelines only focus on hardware and lighting setup. Therefore, most pre- and post-operative images are taken from different perspectives. This makes them not suitable for quantitative analysis such as comparing with simulation results, as it is very difficult to compare paths and distances in two two-dimensional face photos taken from vastly different perspectives. To address this issue, we propose an application to ensure the pre- and post-operative images are taken from the same perspective. We build a mobile application where we first record the face pose of the pre-operative image. When taking the post-operative image, we compare the face pose of the current frame with the pre-operative pose and only take a photo when the difference is below a threshold. We performed a comparison study of taking post-operative images with the proposed application and the phone camera on six head models. Experimental results show that the alignment error for the proposed application is only 1/3 of that of the phone camera, proving the effectiveness of our system.
Purpose: We explain our concept for customization of a guidance instrument, present a prototype, and describe a set of experiments to evaluate its positioning and drilling accuracy.
Methods: Our concept is characterized by the use of bone cement, which enables fixation of a specific configuration for each individual surgical template. This well-established medical product was selected to ensure future intraoperative fabrication of the template under sterile conditions. For customization, a manually operated alignment device is proposed that temporarily defines the planned trajectory until the bone cement is hardened. Experiments (n = 10) with half-skull phantoms were performed. Analysis of accuracy comprises targeting validations and experiments including drilling in bone substitutes.
Results: The resulting mean positioning error was found to be 0.41 ± 0.30 mm at the level of the target point whereas drilling was possible with a mean accuracy of 0.35 ± 0.30 mm.
Conclusion: We proposed a cost-effective, easy-to-use approach for accurate instrument guidance that enables template fabrication under sterile conditions. The utilization of bone cement was proven to fulfill the demands of an easy, quick, and prospectively intraoperatively doable customization. We could demonstrate sufficient accuracy for many surgical applications, e.g., in neurosurgery. The system in this early development stage already outperforms conventional stereotactic frames and image-guided surgery systems in terms of targeting accuracy.
KEYWORDS: Denoising, Optical coherence tomography, Signal to noise ratio, Image classification, Speckle, Medical imaging, Wavelets, Medical diagnostics, Ophthalmology, Classification systems
Noise in speckle-prone optical coherence tomography tends to obfuscate important details necessary for medical diagnosis. In this paper, a denoising approach that preserves disease characteristics on retinal optical coherence tomography images in ophthalmology is presented. We propose semantic denoising autoencoders, which combine a convolutional denoising autoencoder with a priorly trained ResNet image classifier as regularizer during training. This promotes the perceptibility of delicate details in the denoised images that are important for diagnosis and filters out only informationless background noise. With our approach, higher peak signal-to-noise ratios with PSNR = 31.0 dB and higher classification performance of F1 = 0.92 can be achieved for denoised images compared to state-of-the-art denoising. It is shown that semantically regularized autoencoders are capable of denoising retinal OCT images without blurring details of diseases.
In microsurgery, lasers have emerged as precise tools for bone ablation. A challenge is automatic control of laser bone ablation with 4D optical coherence tomography (OCT). OCT as high resolution imaging modality provides volumetric images of tissue and foresees information of bone position and orientation (pose) as well as thickness. However, existing approaches for OCT based laser ablation control rely on external tracking systems or invasively ablated artificial landmarks for tracking the pose of the OCT probe relative to the tissue. This can be superseded by estimating the scene flow caused by relative movement between OCT-based laser ablation system and patient. Therefore, this paper deals with 2.5D scene flow estimation of volumetric OCT images for application in laser ablation. We present a semi-supervised convolutional neural network based tracking scheme for subsequent 3D OCT volumes and apply it to a realistic semi-synthetic data set of ex vivo human temporal bone specimen. The scene flow is estimated in a two-stage approach. In the first stage, 2D lateral scene flow is computed on census-transformed en-face arguments-of-maximum intensity projections. Subsequent to this, the projections are warped by predicted lateral flow and 1D depth flow is estimated. The neural network is trained semi-supervised by combining error to ground truth and the reconstruction error of warped images with assumptions of spatial flow smoothness. Quantitative evaluation reveals a mean endpoint error of (4.7 ± 3.5) voxel or (27.5 ± 20.5) μm for scene flow estimation caused by simulated relative movement between the OCT probe and bone. The scene flow estimation for 4D OCT enables its use for markerless tracking of mastoid bone structures for image guidance in general, and automated laser ablation control.
Pathologies of protective laryngeal reflexes such as the laryngeal adductor reflex (LAR) can increase the risk of aspiration pneumonia, a potentially life-threatening inflammation of the lungs caused by the intrusion of foreign particles into the lower airways. To estimate this risk, a standardized and non-invasive LAR screening method is highly desirable. In previous work, a commercially available high-speed laryngoscope has been equipped with a pressurized fluid system to shoot droplets onto the laryngeal structures for LAR stimulation and subsequent reflex latency evaluation. This Micro-droplet Impulse Testing of the LAR (MIT-LAR) lacks droplet impact site prediction for an unbiased and stimulation site-dependent reflex latency assessment. In this work, a two- phase algorithm leveraging stereoscopic image data for droplet impact site prediction and visualization of this prediction in the laryngoscopic image is proposed. A high-speed stereolaryngoscope requiring only a single camera was designed from scratch by combining two rod lens optics in one common shaft. This setup was used for stereoscopic high-speed image data acquisition of droplets shot at different muzzle energies and endoscope roll angles. Surface reconstruction of the target region is performed using a phantom of the human larynx. The point of intersection between the reconstructed surface and the droplet trajectory approximation recorded previously at known droplet formation parameters is calculated. The proposed approach allows stimulation site prediction for an enhanced MIT-LAR procedure. Quantitative evaluation of impact site prediction accuracy for different droplet muzzle energies and nozzle inclinations yields an average prediction error of 2.9 mm (n = 20).
Projector based augmented reality serves as an alternate visual guidance tool for surgeons when performing complicated open surgeries. In projector based augmented reality, image overlay projection is a technique that allows the surgeon to view the underlying anatomical information such as tissues, tumors etc. directly on the surface of the organ or the patient. This will provide an intuitive view of the surgical navigation data by combining the surgeon’s real world view with the preoperative three dimensional virtual models or instructions. Thus the strain on the surgeon to mentally align and visualize the preoperative data with intraoperative scene is greatly reduced. There are multiple stationary and handheld projectors available in the market today for this purpose. During surgery, stationary projectors mounted on a rack or under the ceiling suffer from a loss of adjustability and further cause shadowing issues when the surgeon occludes the scene. Although hand-held projectors do not have these disadvantages, they have major problems in terms of illuminance and luminous flux. The amount of light at which the hand-held projectors can project virtual additional information on to the patient is very low especially when the surgical lights are switched on. This paper therefore aims to provide an analysis of the requirements for designing such a special hand-held, augmented reality projector system that could be used during surgery, through a user study. Various optical parameters which are a key to design an augmented reality projector such as illuminance, luminance, luminous flux etc. are measured. Apart from that, other parameters such as refresh rate, image size, resolution which are also some important criteria in designing such a special projector, are discussed in this paper with respect to our application.
A common representation of volumetric medical image data is the triplanar view (TV), in which the surgeon manually selects slices showing the anatomical structure of interest. In addition to common medical imaging such as MRI or computed tomography, recent advances in the field of optical coherence tomography (OCT) have enabled live processing and volumetric rendering of four-dimensional images of the human body. Due to the region of interest undergoing motion, it is challenging for the surgeon to simultaneously keep track of an object by continuously adjusting the TV to desired slices. To select these slices in subsequent frames automatically, it is necessary to track movements of the volume of interest (VOI). This has not been addressed with respect to 4DOCT images yet. Therefore, this paper evaluates motion tracking by applying state-of-the-art tracking schemes on maximum intensity projections (MIP) of 4D-OCT images. Estimated VOI location is used to conveniently show corresponding slices and to improve the MIPs by calculating thin-slab MIPs. Tracking performances are evaluated on an in-vivo sequence of human skin, captured at 26 volumes per second. Among investigated tracking schemes, our recently presented tracking scheme for soft tissue motion provides highest accuracy with an error of under 2.2 voxels for the first 80 volumes. Object tracking on 4D-OCT images enables its use for sub-epithelial tracking of microvessels for image-guidance.
Image-to-physical registration based on volumetric data like computed tomography on the one side and intraoperative endoscopic images on the other side is an important method for various surgical applications. In this contribution, we present methods to generate panoramic views from endoscopic recordings for image-to-physical registration of narrow drill holes inside spongy bone. One core application is the registration of drill poses inside the mastoid during minimally invasive cochlear implantations. Besides the development of image processing software for registration, investigations are performed on a miniaturized optical system, achieving 360° radial imaging with one shot by extending a conventional, small, rigid, rod lens endoscope. A reflective cone geometry is used to deflect radially incoming light rays into the endoscope optics. Therefore, a cone mirror is mounted in front of a conventional 0° endoscope. Furthermore, panoramic images of inner drill hole surfaces in artificial bone material are created. Prior to drilling, cone beam computed tomography data is acquired from this artificial bone and simulated endoscopic views are generated from this data. A qualitative and quantitative image comparison of resulting views in terms of image-to-image registration is performed. First results show that downsizing of panoramic optics to a diameter of 3mm is possible. Conventional rigid rod lens endoscopes can be extended to produce suitable panoramic one-shot image data. Using unrolling and stitching methods, images of the inner drill hole surface similar to computed tomography image data of the same surface were created. Registration is performed on ten perturbations of the search space and results in target registration errors of (0:487 ± 0:438)mm at the entry point and (0:957 ± 0:948)mm at the exit as well as an angular error of (1:763 ± 1:536)°. The results show suitability of this image data for image-to-image registration. Analysis of the error components in different directions reveals a strong influence of the pattern structure, meaning higher diversity results into smaller errors.
The introduction of Er:YAG lasers for soft and hard tissue ablation has proven promising results over the last decades due to strong absorption at 2.94 μm wavelength by water molecules. An extension to endoluminal applications demands laser delivery without mirror arms due to dimensional constraints. Therefore, fiber-based solutions are advanced to provide exible access while keeping space requirements to a minimum. Conventional fiber-based treatments aim at laser-tissue interactions in contact mode. However, this procedure is associated with disadvantages such as advancing decrease in power delivery due to particle coverage of the fiber tip, tissue carbonization, and obstructed observation of the ablation progress. The objective of this work is to overcome aforementioned limitations with a customized fiber-based module for non-contact robot-assisted endoluminal surgery and its associated experimental evaluation. Up to the authors knowledge, this approach has not been presented in the context of laser surgery at 2.94 μm wavelength. The preliminary system design is composed of a 3D Er:YAG laser processing unit enabling automatic laser to fiber coupling, a GeO2 solid core fiber, and a customized module combining collimation and focusing unit (focal length of 20 mm, outer diameter of 8 mm). The performance is evaluated with studies on tissue substitutes (agar-agar) as well as porcine samples that are analysed by optical coherence tomography measurements. Cuts (depths up to 3mm) with minimal carbonization have been achieved under adequate moistening and sample movement (1.5mms-1). Furthermore, an early cadaver study is presented. Future work aims at module miniaturization and integration into an endoluminal robot for scanning and focus adaptation.
Laser surgery is an established clinical procedure in dental applications, soft tissue ablation, and ophthalmology. The presented experimental set-up for closed-loop control of laser bone ablation addresses a feedback system and enables safe ablation towards anatomical structures that usually would have high risk of damage. This study is based on combined working volumes of optical coherence tomography (OCT) and Er:YAG cutting laser. High level of automation in fast image data processing and tissue treatment enables reproducible results and shortens the time in the operating room. For registration of the two coordinate systems a cross-like incision is ablated with the Er:YAG laser and segmented with OCT in three distances. The resulting Er:YAG coordinate system is reconstructed. A parameter list defines multiple sets of laser parameters including discrete and specific ablation rates as ablation model. The control algorithm uses this model to plan corrective laser paths for each set of laser parameters and dynamically adapts the distance of the laser focus. With this iterative control cycle consisting of image processing, path planning, ablation, and moistening of tissue the target geometry and desired depth are approximated until no further corrective laser paths can be set. The achieved depth stays within the tolerances of the parameter set with the smallest ablation rate. Specimen trials with fresh porcine bone have been conducted to prove the functionality of the developed concept. Flat bottom surfaces and sharp edges of the outline without visual signs of thermal damage verify the feasibility of automated, OCT controlled laser bone ablation with minimal process time.
Planning and analyzing of surgical interventions are often based on computer models derived from computed tomography images of the patient. In the field of cochlear implant insertion the modeling of several structures of the inner ear is needed. One structure is the overall helical shape of the cochlea itself. In this paper we analyze the cochlea by applying statistical shape models with medial representation. The cochlea is considered as tubular structure. A model representing the skeleton of training data and an atomic composition of the structure is built. We reduce the representation to a linear chain of atoms. As result a compact discrete model is possible. It is demonstrated how to place the atoms and build up their correspondence through a population of training data. The outcome of the applied representation is discussed in terms of impact on automated segmentation algorithms and known advantages of medial models are revisited.
This work proposes new methods for a microstereotactic frame based on bone cement fixation. Microstereotactic frames are under investigation for minimal invasive temporal bone surgery, e.g. cochlear implantation, or for deep brain stimulation, where products are already on the market. The correct pose of the microstereotactic frame is either adjusted outside or inside the operating room and the frame is used for e.g. drill or electrode guidance. We present a patientspecific, disposable frame that allows intraoperative, sterile pose-setting. Key idea of our approach is bone cement between two plates that cures while the plates are positioned with a mechatronics system in the desired pose. This paper includes new designs of microstereotactic frames, a system for alignment and first measurements to analyze accuracy and applicable load.
Recent research has revealed that incision planning in laser surgery deploying stylus and tablet outperforms state-of-the-art micro-manipulator-based laser control. Providing more detailed quantitation regarding that approach, a comparative study of six tablet-based strategies for laser path planning is presented. Reference strategy is defined by monoscopic visualization and continuous path drawing on a graphics tablet. Further concepts deploying stereoscopic or a synthesized laser view, point-based path definition, real-time teleoperation or a pen display are compared with the reference scenario. Volunteers were asked to redraw and ablate stamped lines on a sample. Performance is assessed by measuring planning accuracy, completion time and ease of use. Results demonstrate that significant differences exist between proposed concepts. The reference strategy provides more accurate incision planning than the stereo or laser view scenario. Real-time teleoperation performs best with respect to completion time without indicating any significant deviation in accuracy and usability. Point-based planning as well as the pen display provide most accurate planning and increased ease of use compared to the reference strategy. As a result, combining the pen display approach with point-based planning has potential to become a powerful strategy because of benefiting from improved hand-eye-coordination on the one hand and from a simple but accurate technique for path definition on the other hand. These findings as well as the overall usability scale indicating high acceptance and consistence of proposed strategies motivate further advanced tablet-based planning in laser microsurgery.
This work investigates combination of Optical Coherence Tomography and two cameras, observing a microscopic scene. Stereo vision provides realistic images, but is limited in terms of penetration depth. Optical Coherence Tomography (OCT) enables access to subcutaneous structures, but 3D-OCT volume data do not give the surgeon a familiar view. The extension of the stereo camera setup with OCT imaging combines the benefits of both modalities. In order to provide the surgeon with a convenient integration of OCT into the vision interface, we present an automated image processing analysis of OCT and stereo camera data as well as combined imaging as augmented reality visualization. Therefore, we care about OCT image noise, perform segmentation as well as develop proper registration objects and methods. The registration between stereo camera and OCT results in a Root Mean Square error of 284 μm as average of five measurements. The presented methods are fundamental for fusion of both imaging modalities. Augmented reality is shown as application of the results. Further developments lead to fused visualization of subcutaneous structures, as information of OCT images, into stereo vision.
No investigation is published so far, describing the distortion correction for bent rigid endoscopes. This work comprises
a definition of endoscope bending states and a proof that inhomogeneous (section varying) radial distortion correction
achieves enhanced results in comparison to linear distortion correction. Precaution or advanced distortion correction
techniques should be taken while using bent or deflected endoscopes in applications of computer assisted diagnosis and
therapy.
During endoscopic procedures it is often desirable to determine the distance between anatomical features. One such
clinical application is percutaneous cochlear implantation (PCI), which is a minimally invasive approach to the cochlea
via a single, straight drill path and can be achieved accurately using bone-implanted markers and customized
microstereotactic frame. During clinical studies to validate PCI, traditional open-field cochlear implant surgery was
performed and prior to completion of the surgery, a customized microstereotactic frame designed to achieve the desired
PCI trajectory was attached to the bone-implanted markers. To determine whether this trajectory would have safely
achieved the target, a sham drill bit is passed through the frame to ensure that the drill bit would reach the cochlea
without damaging vital structures. Because of limited access within the facial recess, the distances from the bit to
anatomical features could not be measured with calipers. We hypothesized that an endoscope mounted on a sliding stage
that translates only along the trajectory, would provide sufficient triangulation to accurately measure these distances. In
this paper, the design, fabrication, and testing of such a system is described. The endoscope is mounted so that its optical
axis is approximately aligned with the trajectory. Several images are acquired as the stage is moved, and threedimensional
reconstruction of selected points allows determination of distances. This concept also has applicability in a
large variety of rigid endoscopic interventions including bronchoscopy, laparoscopy, and sinus endoscopy.
The aim of this study is a defined, visually based and camera controlled bone removal by a navigated CO2 laser on the
promontory of the inner ear. A precise and minimally traumatic opening procedure of the cochlea for the implantation of
a cochlear implant electrode (so-called cochleostomy) is intended. Harming the membrane linings of the inner ear can
result in damage of remaining organ functions (e.g. complete deafness or vertigo). A precise tissue removal by a laser-based
bone ablation system is investigated. Inside the borehole the pulsed laser beam is guided automatically over the
bone by using a two mirror galvanometric scanner. The ablation process is controlled by visual servoing. For the
detection of the boundary layers of the inner ear the ablation area is monitored by a color camera. The acquired pictures
are analyzed by image processing. The results of this analysis are used to control the process of laser ablation. This
publication describes the complete system including image processing algorithms and the concept for the resulting
distribution of single laser pulses. The system has been tested on human cochleae in ex-vivo studies. Further
developments could lead to safe intraoperative openings of the cochlea by a robot based surgical laser instrument.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.