Soft tissue augmentation, e.g. for surgical interventions such as liver resection or heart surgery is an extremely complex problem. The superimposition of virtual, geometrically rigit information such as planning trajectories or 3D visualization of anatomical structures from medical imaging data onto the moving and deforming target tissue provides only limited advantages. The geometrical situation of the operation site at the time of image acquisition and preoperative planning has very likely changed when the patient has been moved from the bedding of the CT scanner to the operating table. Also breathing and interaction with surgical instruments changes the anatomical structures around and at the operating site. At maximum, the augmentation of the real operation site with rigit 3D data would allow the surgeon to study ROIs and critical structures within his field of view, however, he can never rely on the correctness of this visualization.
A big step forward to accurate registration of virtual 3D information with the real anatomical structures has been presented by Nazim Haouchine and his fellows with the paper “Single View Augmentation of 3D Elastic Objects” at ISMAR 2014 in Munich.
I’m happy to share with you Nazim’s answers on my questions related to his work.
Can you describe the motivation of your work with respect to medical applications?
My work focuses on the development of an Augmented Reality (AR) framework for minimally invasive liver surgery. This work is motivated by the capability of AR to help surgeons during an operation. Indeed, tumors and vessels computed from pre-operative scans can be superimposed on a video stream per-operatively, which gives an additional support for the surgeon. Current work on medical AR only assume rigid organ motion or suppose that their deformations are negligible. However, when dealing with liver tissue this assumption cannot be considered. Our work focuses on establishing a full AR framework that takes into account elastic organ deformations.
Do you collaborate with doctors and, if yes, how is this collaboration organized?
Yes, we collaborate with doctors. My research group (shacra) is involved in IHU, a research institute dedicated to image-guided surgery, located in Strasbourg hospital, which greatly facilitates collaboration with surgeons. In practice, this collaboration aims to define the real needs of surgeons for Augmented Reality, and to find the best way to involve them in the framework, since we believe that they have an essential role to play. A validation protocol and experimentations are also widely discussed where it remains very challenging to validate the developed methods.
Which accuracy of the deformable registration is sophisticated/enough for liver surgery and other applications?
The actual margins considered for the removal of tumors (in the case of liver surgery) are between 10mm and 25mm around the tumor, We want to reduce the margins as much as possible. Our preliminary results show that with an appropriate biomechanical liver model, these margins can be reduced. Obviously, more validation is needed,
Which data would be augmented at the end in a clinical usage of the system?
The data that have to be augmented are organ internal structures, such as tumors and vascular networks. Indeed, the liver is already visible, so there is no need to superimpose the liver mesh on the laparoscopic images. This is why, we use a biomehcanical model of the liver capable of translating the elastic liver behaviour in order to propagate, in-depth, the surface liver deformations. This results in a correct estimation of tumor positions that can be superimposed. As a clinical usage, if we take the liver resection as an example, where the surgeon removes a part of the liver that contains tumors. The surgeon needs to detect risky areas that could cause bleeding, in that case, augmenting the vessels can be of high benefit.
Can you discuss a little bit the accuracy of simulating soft tissue vs. volume mesh complexity vs. availability of mechanical properties of different soft tissues vs. real-time presentation.
When dealing with medical applications, real-time is a crucial feature to consider, as well as accuracy and stability. My research group Shacra at INRIA, is a multidisciplinary team focussing on the development of real-time soft-tissue simulation using advanced techniques. These works are dedicated to medical simulation, mainly for training and planning. We wanted to bring simulation to the operating room where the need of realistic real-time simulation can be of high benefits. To do so, there is a trade-off to define by choosing the appropriate model that is accurate enough to estimate correct tumor position while ensuring a stable simulation and fast computation. In our recent work, we show that using a heterogeneous model (that contains vessels) driven by visually tracked image-points processed from laparoscopic images, we can satisfy these 3 conditions. Finally, for the availability of mechanical properties of different soft tissues, we believe that integrating elastography data recorded pre-operatively could substantially increase the accuracy of the model. In the meantime, we are considering alternative methods that aim to compute patient-specific models without patient-specific properties.
The following videos show the result of Nazim’s work.