Abstract
Three-dimensional imaging modalities for structural heart disease interventions have become a common feature in the procedural workflow. The images acquired are usually presented on 2D displays, thereby restricting their usefulness and the ability to interact with them. Holographic images created in real time from the volumetric data which float in the air during the procedure, in front of the operator and above the patient, could provide an intuitive and interactive display for the interventionalist and improve procedure outcomes.
Structural heart disease (SHD) is a multifaceted and evolving topic which is constantly recruiting new lesions, imaging techniques and therapeutic options under its broadening wings. Despite the diversity of SHD, a number of common denominators can be identified which emphasise important differences from coronary artery disease (CAD). Whereas CAD relates to “wear and tear” damage whose interventions are “over the wire” in a defined tubular anatomy exterior to the heart, SHD often concerns lesions and defects of ill-defined shape whose interventions are in free three-dimensional (3D) space inside the cardiac chambers. The physician typically has to “build” a personal understanding of the 3D anatomy in his/her head from a series of two-dimensional (2D) images. These factors increase the complexity of interacting with the patient-specific anatomy while performing minimally invasive and transcatheter procedures.
Three-dimensional imaging modalities have the potential to enable and enhance our comprehension, navigation and intervention with spatially complex anatomies and assist in the evaluation of the outcomes of such interventions. These modalities acquire volumetric data to provide high-quality 3D reconstructions, the use of which is becoming more prevalent in clinical settings and workflows. Many SHD procedures, e.g., TAVI, use pre-procedural static imaging by computed tomography (CT) to determine the size and suitability of devices. Real-time 3D transoesophageal echocardiography (TEE) is an integral modality for intraprocedural dynamic imaging for device navigation in atrioventricular valve interventions. Recently, fusion imaging of pre-procedural data with live intraprocedural fluoroscopy and intraprocedural 3D TEE overlaid on live 2D fluoroscopy have been introduced1. Although the quality and definition of 3D images and the delay from acquisition to display continue to improve, the observer’s 3D experience and interaction with the images lag behind since the reconstructions are restricted inside 2D screens, allowing viewing of one plane only.
An attempt to make 3D more tangible has recently become available with rapid prototyping of medical volumetric data from CT or magnetic resonance imaging (MRI)2. A solid model of the imaged anatomy is printed in three dimensions and can be useful for the planning of interventions. However, the disadvantages of this methodology include the provision of only one set of the data in a fixed state and the lack of real-time availability. The need for a 3D live, real-time display which provides unlimited and intuitive interaction with and within the image becomes more important as the demands of the procedures on the SHD interventionalists increase.
Our ability to see an object, understand the spatial inter-relationship of all its features and comprehend its position in space and interact precisely with that object is all due to visual depth perception3. The object’s three dimensionality and distance is perceived owing to the capacity of the visual cortex simultaneously to combine multiple depth cues, including binocular stereopsis, convergence and accommodation. Each eye, since it has a different position in the head, views an object from a slightly different angle and this disparity, stereopsis, is employed by the brain as a cue for depth. Stereoscopy is a method which mimics stereopsis by presenting each eye with two slightly different versions of an image which, when combined in the brain, produce an illusion of depth and three-dimensionality3. The reason for terming this a “3D illusion” is that all the points of light in the stereoscopic image are focused in the same singular focal plane, whereas in an actual object these points arise from multiple different planes and true depth perception has infinite focal planes. The issue of focus becomes critical when bringing the stereoscopic image into close range since, without the capacities of accommodation and convergence, the brain becomes confused, causing visual discomfort and nausea. The “3D” image appears at a different position from its actual source, typically floating in front of the screen or picture, and therefore interacting within the image is challenging since, as the eyes focus on the interacting device, they lose contact with the source and the illusion breaks up. As stated above, even the highest quality stereoscopic image is confined to one focal plane and therefore interactions such as placing a device in the image, or manipulating the image with the observer’s hand inside it, are not possible, since the device or hand occupies multiple focal planes. These, and other, inherent limitations of stereoscopic-based displays restrict their usefulness for image-guided interventions where precise navigation, image interaction and exact tool-tissue interfacing are paramount, especially in close proximity to the operator3.
Holography is a methodology which, using light, creates an exact visual replication of an object in three physical dimensions, including all the depth cues, focal planes and specific co-ordinates4,5. A high-quality hologram should be indistinguishable from the true object it represents. The holographic image occupies an actual space and can be likened to an object created by a 3D printer where the material used to print is light. Since the hologram is actually present, 3D interaction with the image becomes intuitive as with any real object in space and, moreover, head movement and manipulating the image as naturally as a real object will provide further information. These features are not typically possible with a stereoscopic image.
The potential usefulness of a holographic display to medicine in general and, in particular, to performing interventions is clear, especially in complex anatomies. However, the creation of a holographic display is not a straightforward task. Originally, in order to produce holographic images, specialised photographic equipment, film development and dedicated light sources were required4. An adaptation of this process was used in the early 1990s to generate static monochromatic holograms using data from 2D CT images to guide vertebral pedicle screw placement in cadavers6. The first intraoperative use of such holograms was as a guide to assist skull reconstruction by placing the image over the region of surgical interest so that the surgeon could look through the hologram while performing the procedure7. Greyscale holograms of cardiac structures have been created from pre-acquired echocardiographic images of isolated hearts placed on film and viewed in a special viewing box8. These early clinical experiences underscored the anticipated benefits of true depth perception and intuitive comprehension of the spatial anatomy; however, the complexity of producing and displaying the holograms and their being static, monochromatic and acquired pre-procedurally restricted their translation to regular clinical intraprocedural use.
The advent of digital computer-generated holography obviates the need for photography and holographic film by using algorithms to define specific patterns on spatial light modulators which, when coherent light passes through them, will create a specified holographic image5. This methodology has been successfully employed in digital microscopy to create 3D static holograms from 3D volumetric data9. For computer-generated holography to become a clinically useful holographic display the image needs to:
– be free floating in the air within arm’s reach from the observer
– be of adequate 3D size/volume to afford viewing of the details of anatomical structures
– have a wide angle of view to allow freedom of movement of the observer
– be high quality with no loss or compression of data from that acquired by the modality
– be dynamic for moving structures such as the heart and valves
– be in real time to provide intraprocedural navigation and intervention
– be easy to manipulate to fit in to the clinical workflow
– not require ancillary equipment such as goggles which isolate operator from environment.
The authors are currently involved in evaluating the feasibility and usefulness of creating holograms using a prototype holographic display and interface system (RealView Imaging Inc., Yokneam, Israel) that projects high-resolution 3D holographic images “floating in the air” without the need for any human-mounted device or goggles. The holograms are generated in real time and allow the user to interact with the digital data by inserting a tool or hand into the image space (Figure 110, Moving image 1). Holography of real-time 3D volumetric medical data could provide the intuitive display which is acutely needed to guide the more complex structural heart interventions. Pertinent examples include atrioventricular valve interventions, mitral valve clipping and annuloplasty and tricuspid annuloplasty, left atrial appendage closure and anatomical arrhythmia ablations. The procedural success and long-term outcomes of these types of intervention are directly influenced by the accuracy of the tissue-tool interface. The exact positioning of a mitral clip in the centre of the regurgitant jet while attempting to engage the appropriate segments of the anterior and posterior mitral valve leaflets has been facilitated by 3D real-time TEE1. Similarly, 3D electroanatomical mapping systems have become an essential apparatus for arrhythmia ablations, improving procedure outcomes while reducing radiation exposure11.
Figure 1. Holograms created by RealView from volumetric data of a virtually deployed CoreValve® (Medtronic, Minneapolis, MN, USA) simulated with the TAVIguide™ technology (FEops, Ghent, Belgium). The TAVIguide technology predicts how a certain TAVI device will interact with a specific patient. Based on these insights, the physician gets a better view of the device host interaction, prior to the intervention, thus allowing optimal personalised device selection and positioning10. A) The partially deployed CoreValve is seen by slicing into the ascending aorta. B) The interaction with the native aortic leaflets can be seen from any angle. C) The observer’s index finger rotates the aorta and fully deployed CoreValve. Please note that holograms cannot be fully appreciated on a 2D screen, but need to be observed directly.
The experience of true depth perception in a live 3D medical hologram in the catheterisation laboratory could have significant ramifications for guidance of interventions in free 3D space. When they become available, prospective studies will be needed to evaluate the impact of 3D holographic displays in the clinical setting by comparing procedure-related parameters, such as length, outcome and radiation exposure, to those achieved when using standard 2D displays alone, in procedures during which real-time 3D imaging is an integral part of the workflow.
Conflict of interest statement
E. Bruckheimer and C. Rotschild are stockholders and employees of RealView Imaging.
Supplementary data
Moving image 1. Dynamic hologram.
Supplementary data
To read the full content of this article, please download the PDF.
Dynamic hologram.
