Soft Tissue Augmentation – Interview with Nazim Haouchine

Soft tissue augmentation, e.g. for surgical interventions such as liver resection or heart surgery is an extremely complex problem. The superimposition of virtual, geometrically rigit information such as planning trajectories or 3D visualization of anatomical structures from medical imaging data onto the moving and deforming target tissue provides only limited advantages. The geometrical situation of the operation site at the time of image acquisition and preoperative planning has very likely changed when the patient has been moved from the bedding of the CT scanner to the operating table. Also breathing and interaction with surgical instruments changes the anatomical structures around and at the operating site. At maximum, the augmentation of the real operation site with rigit 3D data would allow the surgeon to study ROIs and critical structures within his field of view, however, he can never rely on the correctness of this visualization.

A big step forward to accurate registration of virtual 3D information with the real anatomical structures has been presented by Nazim Haouchine and his fellows with the paper “Single View Augmentation of 3D Elastic Objects” at ISMAR 2014 in Munich.

Soft Tissue Augmented Reality

Image courtesy of Nazim Haouchine

The following videos show the result of Nazim’s work.

I’m also happy to share with you Nazim’s answers on my questions related to his work.

Can you describe the motivation of your work with respect to medical applications?
My work focuses on the development of an Augmented Reality (AR) framework for minimally invasive liver surgery. This work is motivated by the capability of AR to help surgeons during an operation. Indeed, tumors and vessels computed from pre-operative scans can be superimposed on a video stream per-operatively, which gives an additional support for the surgeon. Current work on medical AR only assume rigid organ motion or suppose that their deformations are negligible. However, when dealing with liver tissue this assumption cannot be considered. Our work focuses on establishing a full AR framework that takes into account elastic organ deformations.

Do you collaborate with doctors and, if yes, how is this collaboration organized?
Yes, we collaborate with doctors. My research group (shacra) is involved in IHU, a research institute dedicated to image-guided surgery, located in Strasbourg hospital, which greatly facilitates collaboration with surgeons. In practice, this collaboration aims to define the real needs of surgeons for Augmented Reality, and to find the best way to involve them in the framework, since we believe that they have an essential role to play. A validation protocol and experimentations are also widely discussed where it remains very challenging to validate the developed methods.

Which accuracy of the deformable registration is sophisticated/enough for liver surgery and other applications?
The actual margins considered for the removal of tumors (in the case of liver surgery) are between 10mm and 25mm around the tumor, We want to reduce the margins as much as possible. Our preliminary results show that with an appropriate biomechanical liver model, these margins can be reduced. Obviously, more validation is needed,

Which data would be augmented at the end in a clinical usage of the system?
The data that have to be augmented are organ internal structures, such as tumors and vascular networks. Indeed, the liver is already visible, so there is no need to superimpose the liver mesh on the laparoscopic images. This is why, we use a biomehcanical model of the liver capable of translating the elastic liver behaviour in order to propagate, in-depth, the surface liver deformations. This results in a correct estimation of tumor positions that can be superimposed. As a clinical usage, if we take the liver resection as an example, where the surgeon removes a part of the liver that contains tumors. The surgeon needs to detect risky areas that could cause bleeding, in that case, augmenting the vessels can be of high benefit.

Can you discuss a little bit the accuracy of simulating soft tissue vs. volume mesh complexity vs. availability of mechanical properties of different soft tissues vs. real-time presentation.
When dealing with medical applications, real-time is a crucial feature to consider, as well as accuracy and stability. My research group Shacra at INRIA, is a multidisciplinary team focussing on the development of real-time soft-tissue simulation using advanced techniques. These works are dedicated to medical simulation, mainly for training and planning. We wanted to bring simulation to the operating room where the need of realistic real-time simulation can be of high benefits. To do so, there is a trade-off to define by choosing the appropriate model that is accurate enough to estimate correct tumor position while ensuring a stable simulation and fast computation. In our recent work, we show that using a heterogeneous model (that contains vessels) driven by visually tracked image-points processed from laparoscopic images, we can satisfy these 3 conditions. Finally, for the availability of mechanical properties of different soft tissues, we believe that integrating elastography data recorded pre-operatively could substantially increase the accuracy of the model. In the meantime, we are considering alternative methods that aim to compute patient-specific models without patient-specific properties.

Christoph Bichlmeier

Enthusiast of Augmented Reality for Medical Applications.

More Posts - Website

Follow Me:

Exhibition Combining an Anatomical Wax Cabinet and Art

You are looking for inspiration to design the Augmented Reality based view into the body and visualize anatomy? Then take a look at the hostorical approaches how to achieve appealing, instructive views into the human body.

Recently, the exhibition Blicke ! Körper ! Sensationen ! Ein anatomisches Wachskabinett und die Kunst (Views! Bodies! Sensations! An anatomical wax cabinet and the art) in Dresden, Germany has opened its doors recently.

The exhibition is based on a collection that has been established around 1900 in Dresden and is presented for the first time in a museeum. For centuries these pieces were shown at fun fairs to educate the audience about health care but also for pure amusement reasons. (see also the press release, which is in German only)

You will be able to visit the exhibition from 11th of Oct. 2014 until 19th of April 2015 at the Deutschen Hygiene-Museum in Dresden.

Learning from Artists How to Design the Immersive View

When having started developing Augmented Reality views into the body, I got a lot inspired by artists, who have designed those views using different media formats. One of the most exiting artists in this field is Alexander Tsiaras, Founder, CEO and Editor-in-Chief of doing beautiful illustrations to demonstrate the interior of the human body.

Recently, I just stumbled upon an image of a guy having his entire head covered with an anatomical tattoo showing parts of the skull, the brain and muscles. While this is an exeptional statement to tell the public that he fell in love with the subject anatomy, I found that it was definitively great artwork. A less durable approach is presented by Danny Quirk, a body painting artist of Springfield, MA, USA, who “creates body paintings with latex, markers and some acrylic that appear as if his models’ skin is peeled back.” (see full article at

Here is another example of anatomical body as part of the traveling exhibition Body Worlds.

Transfering the artists knowledge and skills about the correct design of immersive views into the body to computer graphics and image compositions of Augmented Reality scenes  is certainly an important task to create apealing as well as useful Augmented Reality experiences in the health care sector. Of course, these visualizations need to be adapted to the  designated application such as teaching and training or surgical navigation.

Christoph Bichlmeier

Enthusiast of Augmented Reality for Medical Applications.

More Posts - Website

Follow Me: