Analyzing Natural Language Commands during Laparoscopic Surgery for Robotic Camera Guidance

In this work, we investigate the use of natural language commands for adjusting camera positions during laparoscopic surgery in order to equip a robotic assistance systems with an intuitive voice interface. The integration of natural language commands enables the surgeon to flexibly adapt the camera’s position and to provide feedback to a learning assistance system. Based on the transcripts from 20 laparoscopic surgeries the verbal commands were clustered to receive a grounded understanding of the different command types used. These insights then serve as input for supervised classification methods that map natural language commands to actions of the robotic assistance system that guides the camera. The preliminary results show the need for more complex voice command structures than those presently integrated in existing assistance systems.