[2208.08880] STTAR: Surgical Tool Tracking using off-the-shelf Augmented Reality Head-Mounted Displays

The use of Augmented Reality (AR) for navigation purposes has shown
beneficial in assisting physicians during the performance of surgical
procedures. These applications commonly require knowing the pose of surgical
tools and patients to provide visual information that surgeons can use during
the task performance. Existing medical-grade tracking systems use infrared
cameras placed inside the Operating Room (OR) to identify retro-reflective
markers attached to objects of interest and compute their pose. Some
commercially available AR Head-Mounted Displays (HMDs) use similar cameras for
self-localization, hand tracking, and estimating the objects’ depth. This work
presents a framework that uses the built-in cameras of AR HMDs to enable
accurate tracking of retro-reflective markers, such as those used in surgical
procedures, without the need to integrate any additional components. This
framework is also capable of simultaneously tracking multiple tools. Our
results show that the tracking and detection of the markers can be achieved
with an accuracy of 0.09 +- 0.06 mm on lateral translation, 0.42 +- 0.32 mm on
longitudinal translation, and 0.80 +- 0.39 deg for rotations around the
vertical axis. Furthermore, to showcase the relevance of the proposed
framework, we evaluate the system’s performance in the context of surgical
procedures. This use case was designed to replicate the scenarios of k-wire
insertions in orthopedic procedures. For evaluation, two surgeons and one
biomedical researcher were provided with visual navigation, each performing 21
injections. Results from this use case provide comparable accuracy to those
reported in the literature for AR-based navigation procedures.