Real-time surface reconstruction from stereo endoscopic images for intraoperative registration

Minimally invasive surgery is a medically complex discipline that can heavily benefit from computer assistance. One way to assist the surgeon is to blend in useful information about the intervention into the surgical view using Augmented Reality. This information can be obtained during preoperative planning and integrated into a patient-tailored model of the intervention. Due to soft tissue deformation, intraoperative sensor data such as endoscopic images has to be acquired and non-rigidly registered with the preoperative model to adapt it to local changes. Here, we focus on a procedure that reconstructs the organ surface from stereo endoscopic images with millimeter accuracy in real-time. It deals with stereo camera calibration, pixel-based correspondence analysis, 3D reconstruction and point cloud meshing. Accuracy, robustness and speed are evaluated with images from a test setting as well as intraoperative images. We also present a workflow where the reconstructed surface model is registered with a preoperative model using an optical tracking system. As preliminary result, we show an initial overlay between an intraoperative and a preoperative surface model that leads to a successful rigid registration between these two models.