Augmented reality, superhuman abilities and the future of medicine – OrthoFeed

Earlier this month, I participated as a panelist at the Digital Orthopedics Conference in San Francisco (DOCSF 2022) where a major theme was to imagine the medical profession in the year 2037. In preparation for the event, a small group of us reviewed the latest research on the clinical uses of virtual and augmented reality and critically assessed the current state of the field.

I have to admit, I was deeply impressed by how far augmented reality (AR) has progressed over the last eighteen months for use in medicine. So much so, that I don’t expect we’ll need to wait until 2037 for AR to have a major impact on the field. In fact, I predict that by the end of this decade augmented reality will become a common tool for surgeons, radiologists, and many other medical professionals.  And by the early 2030s, many of us will go to the family doctor and be examined by a physician wearing AR glasses.

Augmented reality will give doctors superpowers.  

I’m talking about superhuman capabilities for visualizing medical images, patient data, and other clinical content. The costs associated with these new capabilities are already quite reasonable and will decrease rapidly as augmented reality hardware gets produced in higher volumes in the coming years.

The first superpower is x-ray vision.

Augmented reality will give doctors the ability to peer directly into a patient and see evidence of trauma or disease at the exact location in their body where it resides.  Of course, the ability to look under the skin already exists with tools like CT and MRI scanning, but currently, doctors view these images on flat screens and need to imagine how the images relate to the patient on the table. This type of mental transformation is an impressive skill, but it takes time and cognitive effort, and is not nearly as informative as it would be if doctors could simply gaze into the human body.

With AR headsets and new techniques for registering 3D medical images to a patient’s real body, the superpower of x-ray vision is now a reality. In an impressive study from Teikyo University School of Medicine in Japan, an experimental emergency room was tested with the ability to capture whole-body CT scans of trauma patients and immediately allow the medical team, all wearing AR headsets, to peer into the patient on the exam table and see the trauma in the exact location where it resides.  This allowed the team to discuss the injuries and plan treatment without needing to refer back and forth to flat screens, saving time, reducing distraction, and eliminating the need for mental transformations.

In other words, AR technology takes medical images off the screen and places them in 3D space at the exact location where it’s most useful to doctors – perfectly aligned with the patient’s body.  Such a capability is so natural and intuitive, that I predict it will be rapidly adopted across medical applications. In fact, I expect that in the early 2030s doctors will look back at the old way of doing things, glancing back and forth at flat screens, as awkward and primitive.