Cong Gao - JHU Computer Science - Johns Hopkins University

Fluoroscopic Navigation for Continuum Manipulator

- TBME 2021 paper - IPCAI 2019 paper - Oral slide - Recorded oral presentation

Brief Intro: We present an image-based navigation solution for a surgical robotic system with a Continuum Manipulator~(CM). Our navigation system uses only fluoroscopic images from a mobile C-arm to estimate the CM shape and pose with respect to the bone anatomy. The CM pose and shape estimation is achieved using image intensity-based 2D/3D registration. A learning-based framework is used to automatically detect the CM in X-ray images, identifying landmark features that are used to initialize and regularize image registration. We also propose a modified hand-eye calibration method that numerically optimizes the hand-eye matrix during image registration. The proposed navigation system for CM positioning was tested in simulation and cadaveric studies.

ProST: Projective Spatial Transformers

- MICCAI2020 - Oral Video - Github

Brief Intro: We propose a novel Projective Spatial Transformer module that generalizes spatial transformers to projective geometry, thus enabling differentiable volume rendering. We demonstrate the usefulness of this architecture on the example of 2D/3D registration between radiographs and CT scans. Specifically, we show that our transformer enables end-to-end learning of an image processing and projection model that approximates an image similarity function that is convex with respect to the pose parameters, and can thus be optimized effectively using conventional gradient descent.

Robot-Assisted Femoroplasty

- TMRB paper - SPIE2020 paper - Oral slide - Recorded oral presentation

Brief Intro: Femroplasty is a proposed therapeutic method for preventing osteoporotic hip fractures in elderly. Patient-specific femoroplasty requires accurate 3D pose estimation of the proximal femur and injection device positioning. We proposed a fiducial-free 2D/3D registration method for robot-assisted femoroplasty system navigation.

SyntheX

- ArxiV Preprint

Brief Intro: We proposed a synthetic training data generation pipeline for machine learning X-ray image analysis tasks, which we refer to as SyntheX. We demonstrated the clinical applications of SyntheX on two downstream tasks, namely hip imaging and surgical robotic tool detection. Through conducting precisely controlled experiments on a unique pelvic benchmark dataset, we isolated and quantified the role of domain shift in the deterioration of machine learning model performance from training in simulation to deployment on real data.

Autonomous Spinal Robot

- Global Spine Journal

Brief Intro: We present an autonomous robotic spine needle injection system using fluoroscopic image-based navigation. Our system includes patient-specific planning, intra-operative image-based 2D/3D registration and navigation, and automatic robot-guided needle injection. We performed intensive simulation studies to validate the registration accuracy. During injections, all needle tips were placed within the defined safety zones for this application. The results suggest the feasibility of using our image-guided robotic injection system for spinal orthopedic applications.

Surgical Force Prediction

- MICCAI 2018 CARE workshop paper - Best Paper Award! (second place) - Oral slide - Github

Brief Intro: Robotic surgery has been proven to offer clear advantages during surgical procedures, however, one of the major limitations is obtaining haptic feedback. Since it is often challenging to devise a hardware solution with accurate force feedback, we propose the use of “visual cues” to infer forces from tissue deformation. We employ deep learning to infer forces from video as an attractive low-cost and accurate alternative to typically complex and expensive hardware solutions with RGB-Point Cloud Temporoal Convolutional Networks.