News — “It’s not like changing the alternator in your car where you can easily reference the repair manual or a technical read-out in the moment,” explained Dr. Stefan Johnson of teaching surgery to medical students and residents. “We’re still teaching with a mentor and pointing out things with a forceps or surgical instrument and saying ‘cut here’ or ‘watch me do this and then you repeat the maneuver.’”
Laparoscopy and robotics notwithstanding, this is how surgical training has been done for centuries, said Johnson.
Thanks to a partnership between the UND’s School of Medicine & Health Sciences (SMHS) Department of Surgery and the School of Electrical Engineering and Computer Sciences (SEECS) within the UND College of Engineering and Mines, though, this age-old method of surgical education is getting an update.
Over the past eighteen months, faculty from the two UND teams have been helping the Surgery Residency Program that Johnson directs experiment with augmented reality (AR) technology in the surgical suite.
Not quite virtual reality, the AR concept uses enhanced glasses that allow for supplemental images or videos to overlay what the physician sees in the operating room (OR). These images or videos can be anything, Johnson said, from radiological images like X-rays or CT scans to patients’ vital signs, instructional videos, and even pages from the surgery textbook, all passed in front of surgeons’ field of vision.
UND leading the world
“So, in the old days, residents would essentially practice on cadavers and patients—they didn’t have much choice,” continued Johnson, who explained that where patient safety is concerned, the real OR is hardly the place for cumbersome, and potentially unclean, medical gear, textbooks, or video monitors. “But this technology has potential to bring better knowledge into the education environment, and for clinical use in general, safely.”
The AR glasses concept was developed by former SMHS clinical professor of surgery Mark Jensen, M.D., who helped UND secure a $20,000 National Institutes of Health SHARPhub grant to study the concept. Although he retired before the grant period ended, Jensen said the fact that nothing has, to date, been published in the medical literature on AR in the surgical suite means UND has an opportunity to be a world-leader.
Enter Ron Marsh, Ph.D., professor in the SEECS who knows Jensen from their mutual interest in ham radio and asked Jensen how his team could help.
“I have a student doing research in virtual reality,” said Marsh whose team is handling the software side of the project. “We looked at some of the different glasses and settled on the set that we’re using now. Then, we were in contact with the manufacturer and discovered that they had developed a Zoom application that works on these glasses. The Zoom capability gives us everything we want, so we basically we took over figuring out how you get the AR part to work from the technical standpoint.”
If the software and wireless connectivity issues can be ironed out, said Marsh, this relatively inexpensive technology – $500 per pair of glasses – could radically improve the practice of not only clinical surgery but rural medicine.
“Let’s say you’re a doctor in Cando, which means you’re maybe not a trauma surgeon,” Marsh mused. “And somebody’s brought in who’s critically injured. Now Cando is pretty rural. If they have a pair of these glasses, and the trauma center in Fargo has a pair of glasses, with Zoom in these glasses the two providers can connect and the doc in Cando can look down at the patient – there’s a camera in the glasses – and the surgeon in Fargo could literally see what his colleagues see.”
Lifesaving technology
In effect, said Marsh, the “active” space on the AR lenses would allow the surgeon in Fargo to advise the physician in Cando by writing or mapping out, with a stylus, instructions on the Cando physician’s field of vision in real-time – cut here, suture there – from Fargo.
“If you’re in the military or maybe with first responders in rural areas, it gives you a truly interactive telemedicine for these emergency surgery situations,” said Marsh. “This should be really exciting, because in traumatic situations I think this could save lives.”
Before such technology is deployed in the field, though, Johnson said that his vision is to get to a point where preceptors can provide such real-time instructions and feedback to learners through the glasses in the educational setting.
“It’s wide open,” Johnson continued. “You could do this in the anatomy lab, maybe put images from the surgery textbook into the glasses as students practice these operations on cadavers. You couldn’t do that in a sterile OR. We can call up any image we want in the glasses.”
And with UND leading the way, operating rooms across the region may be able to do just that.
“We now have capabilities of creating 3D reconstructions of imaging from CT scans that detail anatomical relationships of blood vessels and the location of tumors,” added Dr. Sabha Ganai, an associate professor of surgery for the SMHS who specializes in surgical oncology, hepatobiliary surgery, and endocrine surgery at Sanford Health in Fargo, N.D. “What is nice about AR glasses is it allows us the ability to superimpose the visualization of prior imaging and real time ultrasounds to help guide doing complex therapies like microwave ablations. We currently have to turn our head while we are working, but the future is an integration of our actions with what we see ahead of us.”
RSS