A voice-activated augmented reality headset aided in successful shoulder surgery at California’s Scripps Clinic, a first in the region. The three-dimensional holographic representation of the surgical plan projected onto the patient and modified by voice commands improves the surgeon’s precision and the outcome of the operation and could become a common element in all kinds of medical procedures in the near future.
Improved Surgical Aid
Wearing the headset as seen above during the procedure allowed the surgeon to view and verify the patient’s preoperative plan, which was pre-constructed from CT scans. The model is placed over the patient without obstructing the surgeon’s view of the patient. The surgeon’s voice commands allow them to rotate and zoom in and out without using their hands, which is critical during surgery. Scripps joins nearly three dozen healthcare providers in the US who are using the headset for shoulder replacement surgery. That number is rising rapidly, as it’s only been two years since a Minnesota surgeon first used the AR headset and operating system.
“Being able to view the full surgical plan during surgery is a great benefit to patients because it can help surgeons accurately replicate the preoperative plan,” says Dr. Brian Rebolledo, the Scripps orthopedic surgeon who performed the procedure. “Having a detailed 3D model of the patient’s surgical plan right in our sights in real time opens a new window to help further improve the procedure.”
Augmented reality in healthcare and business services is increasing as visual and speech technology improves. For example, Virti uses virtual being developer Virti a digital environment until training doctors to better communicate using AI-powered virtual patients. The federal government has also invested in AR technology. Xerox scored a contract with DARPA to create an augmented reality system managed by an interactive AI teacher, while Microsoft won $22 billion US military contract to produce improved versions of its Hololens 2 smart glasses. The Integrated Visual Augmentation System (IVAS) devices provide enhanced visibility and information about the soldier’s environment, with a built-in voice assistant and tactile controls to operate the custom headset.