Image-guided Navigation
The use of minimally invasive and flexible access surgery has imposed significant challenges on surgical navigation as the operator can no longer have direct access to the surgical site with unrestricted vision, force and tactile feedback. How to combine prior knowledge of the anatomical model with subject specific information derived from pre- and intra-operative imaging is a significant research challenge. Furthermore, effective surgical guidance is essential to the clinical application of dynamic active constraints for robotically assisted MIS and the development of new surgical tools for emerging flexible access surgical approaches such as Natural Orifice Transluminal Endoscopic Surgery (NOTES) and Single Incision Laparoscopic Surgery (SILS).
For image-guided surgery, another key requirement is the augmentation of the exposed surgical view with pre- or intra-operatively acquired images or 3D models. The inherent challenge is accurate registration of pre- and intra-operative data to the patient, especially for soft tissue where there is large-scale deformation. With the increasing use of intra-operative imaging techniques, our work has been focussed on the development of high- fidelity Augmented Reality (AR) techniques combined with real-time surgical vision and new robotic instruments for further enhancing the accuracy, safety, and consistency of MIS procedures.
Research themes
Research themes
Surgical workflow analysis
Surgical episode segmentation based on vision, instrument usage and kinematics, and video-oculography (eye tracking); workflow recovery for intra-operative CAD (computer aided decision support) and machine learning; analysis and modelling of surgical workflow with consideration of perceptual and cognitive factors, as well as team interaction for detecting dis-orientation, hesitation and precursors to surgical errors.
3D tissue deformation recovery
Real-time tissue deformation recovery based on computer vision by combing multiple depth cues with statistical shape modes and linear/non-linear shape instantiation techniques. Image Constrained Biomechanical Models – inverse finite element modelling based on anatomical constraints derived from surface fiducials or intra-operative imaging data, incorporating tissue feature tracking, spatiotemporal registration, real-time finite element simulation, force recovery and AR visualisation for surgical guidance.
Augmented reality visualisation
High fidelity AR visualisation based on our patented technology on ‘inverse realism’ and real-time instrument tracking for intra-operative imaging probes (e.g., pCLE, OCT and US).
Dynamic active constraints
Patient specific model generation, adaptation and real-time proximity query, haptic rendering for prescribing dynamic active constraints during robotically assisted MIS.