We have developed a 5.5mm and 10mm dual optical channel laparoscope that combines both exit channels into a
single, standard, endoscopic eye cup which attaches directly to a single, conventional HD camera head. We have also
developed image processing software that auto-calibrates, aligns, enhances and processes the image so that it can be
displayed on a stereo/3D display to achieve a true 3D effect.
The advantages to the end user for such a 3D system are that they do not have to purchase a new camera system, all of
their existing scopes are still available to use, as are all integrated OR features. They will be able to add 3D capability to
current HD system by purchasing only stereo scopes and a small video processing computer box and adding a 2D/3D
HD capable monitor.
Gathering depth information through an endoscope or laparoscope during surgical or other procedures is quite difficult. There are stereo laparoscopes but generating three-dimensional models with them is very difficult. Accurate real-time generation of three-dimensional models through a laparoscope is a needed technology to enable a wide range of surgical applications. We have designed a miniature laparoscopic optical system consisting of a single laser whose pattern is modulated and uses the laparoscope as the optical display path into the body. Two cameras, one sensitive to the laser light and the other for full color imaging share this same tube as the laser projector but use the light from the opposite direction. The images gathered by the laser sensitive camera are used to generate a three dimensional map, and the color image is used to acquire the corresponding texture map. High-speed image processing hardware is used to generate 3D information using a structured light technique. The user can then re-render the acquired scene in 3D. The optical system is divided into a removable upper half consisting of the cameras, laser, digital light switches and combining optics. The lower half is the laparoscope or endocope that can be sterilized. There can be several variations in the configuration of the laparoscope optical half that tailor to different procedures.
Creation of accurate surface models of abdominal organs is essential for many developing technologies in medicine and surgery. One application we are working towards is augmented reality (AR) visualization for laparoscopic surgery. Our current system meets some, but not all, of the requirements. We use two custom built laparoscopes, a custom built miniature projector, a standard camera, and a standard video capture and processing card to implement a laparoscopic structured light range acquisition system. We will briefly show the custom hardware but will emphasize the structured light depth extraction techniques used for the unique properties of surfaces inside the body, particularly dealing with specular reflections. In early experiments, we studied the effectiveness of our algorithm in highly specular environments by creating range images acquired from fresh animal organs. These experiments used a large projector, open abdomens, and offline image processing. We report the results of experiments using our miniature projector, and on line processing.
Gathering depth data using structured light has been a procedure for many different environments and uses. Many of these system are utilized instead of laser line scanning because of their quickness. However, to utilize depth extraction for some applications, in our case laparoscopic surgery, the depth extraction must be in real time. We have developed an apparatus that speeds up the raw image display and grabbing in structured light depth extraction from 30 frames per second to 60 and 180 frames per second. This results in an updated depth and texture map of about 15 times per second versus about 3. This increased update rate allows for real time depth extraction for use in augmented medical/surgical applications. Our miniature, fist-sized projector utilizes an internal ferro-reflective LCD display that is illuminated with cold light from a flex light pipe. The miniature projector, attachable to a laparoscope, displays inverted pairs of structured light into the body where these images are then viewed by a high-speed camera set slightly off axis from the projector that grabs images synchronously. The images from the camera are ported to a graphics-processing card where six frames are worked on simultaneously to extract depth and create mapped textures from these images. This information is then sent to the host computer with 3D coordinate information of the projector/camera and the associated textures. The surgeon is then able to view body images in real time from different locations without physically moving the laparoscope imager/projector, thereby, reducing the trauma of moving laparoscopes in the patient.
Head mounted displays (HMDs) have disappointed real world users in their inability to live up to over-hyped expectations. This does not, however, mean that HMDs are useless. While still technologically lacking in some areas, appropriately designed HMDs can be extremely useful tools. We will look at the limitations of current HMDs and ways around them. Rather than approach the problem from the optical, electrical and mechanical engineer's point of view, we will approach it from the physiology point of view, answering the question; what is needed to create a useful HMD. The paper is divided into two separate sections. The first, is a description of the performance of the human visual system. The second, addresses how designers attempt to mimic the human visual system in an HMD. This second section will discuss applications that need the specific performance described in section one, current solutions to those needs and finally ideal solutions not yet implemented. Finally, a summary of these findings is presented in a table format.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.