SAN DIEGO—From computer vision tools designed to increase patient safety to 3D models of organs that can be used to inform surgical decision making, various experts explored how new and powerful innovations, augmented reality and artificial intelligence are poised to improve the surgical landscape in a session at the American College of Surgeons 2022 Clinical Congress.
Richard M. Satava, MD, FACS, a professor emeritus of surgery at the University of Washington, in Seattle, kicked off the discussion by explaining how advancements in infrastructure are moving medicine forward through many channels.
According to Dr. Satava, our growing telecommunications infrastructure will be a fundamental change for healthcare. Fifth-generation (5G) systems are the newest hardware upgrade that will enhance capabilities among and within hospital systems.
“We need high bandwidth and we need low latency,” Dr. Satava said. Latency is a delay from the time that a surgeon moves their hand to the time the robotic instrumentation follows the movement. Dr. Satava explained that the low latency (1-10 milliseconds) of 5G networks is a major factor in the advancement of telesurgery’s feasibility.
“As time goes by, we will be able to do this globally without perceivable latency,” he said.
Dr. Satava also discussed several other basic infrastructure technologies that will contribute to the future healthcare ecosystem, including Internet of Things devices, big data and analytics, supercomputing, and edge computing.
Edge computing, which mediates connections between the cloud, telecommunications and information resources, is going to be a “huge enabler,” he said. “Whether or not your hospital has artificial intelligence or a supercomputer won’t matter, because edge computing will make that resource available.”
Dr. Satava noted that the communication ecosystem has evolved from one-to-one communication, for example, between two telephones, to multiple-to-multiple communications, where anything can talk to anything. However, resistance to new technologies is a hurdle that must be overcome.
“Inertia is our No. 1, compliance is our No. 2 problem,” he said. “As we move forward, that barrier is not going to change.”
Real-Time Holographic Navigation
Yu Saito, MD, PhD, FACS, the senior lecturer at Tokushima University, in Japan, described the utility of new visualization tools for planning and performing hepatobiliary pancreatic (HPB) surgery.
Dr. Saito explained how the Holoeyes program (www.holoeyes.jp/en) allows patient imaging data to be used to generate 3D models of organs that can be used to inform surgical decision making.
“It takes only five minutes from the upload of the data until the generation of an application,” he noted.
The resulting 3D model can be installed into head-mounted displays for preoperative simulation or intraoperative support. Dr. Saito described patient cases showcasing the utility of these tools.
In one case of a patient with 21 colorectal liver metastases, preoperative simulation allowed the headset operator to virtually immerse themselves into the patient’s liver to better visualize tumor locations. An audience was able to follow along by viewing the simulation on a separate screen. Intraoperative ultrasound was then used to generate a hologram to assist with surgical navigation for the patient.
For this complex operative procedure, successful resections were performed “without any postoperative complications, and the patient has not had any recurrence after surgery for two years,” Dr. Saito said.
In another case, intraoperative use of a shared hologram overlay helped surgeons successfully navigate an anatomic anomaly to safely occlude hepatic inflow. Similarly, in biliary surgery, shared holograms facilitated multiuser intraoperative use of 3D cholangiograms to explore detailed information about patient’s bile duct anatomy and could be combined with indocyanine green (ICG) navigation.
“Holographic navigation could be useful in terms of image sharing, spatial recognition and real time for HPB surgery,” Dr. Saito concluded.
Surgical AI and Error Prevention
Daniel Hashimoto, MD, FACS, an assistant professor of surgery at the University of Pennsylvania Perelman School of Medicine, in Philadelphia, shared his experience with a variety of new AI tools and explained how the tools could assist surgeons and increase safety across different procedures.
Dr. Hashimoto highlighted how computer vision tools can be used intraoperatively to enhance and support surgeons’ skills. One model he shared can characterize the surgical field by automatically detecting surgical steps, and other tools can identify safe or unsafe dissection zones, detect components of the critical view of safety, and guide resection margins.
Machine learning also can be used to characterize tissue properties, Dr. Hashimoto illustrated. By training an AI system to understand the relationship between ICG decay times in healthy versus malignant tissues, surgeons can use the AI model to support intraoperative decision-making.
“Even if this liver tissue looks normal, I can look at the decay and run an algorithm and say that the likelihood that there is tumor in there is higher, so we should probably resect this part of the liver as well,” he explained. “It’s a really creative way to use machine learning to understand that fluoroscopic data.”
Dr. Hashimoto also noted that, in addition to intraoperative support, AI tools can characterize the behavior of the surgical staff to provide quality improvement information.
“Ultimately the idea is not really to replace us as surgeons, but it is to augment our abilities to make decisions,” Dr. Hashimoto said.
Dr. Hashimoto shared several obstacles that must be overcome if computer vision will move forward as a powerful surgical innovation. Fears of litigation, laws and regulations, and limited resources are among the behavioral obstacles, and low-quality, insufficient, or biased data and computational expenses represent technical/operational hurdles.
“We’re hoping that we can try to move the cultural barriers around collecting these data and building out these types of technologies,” Dr. Hashimoto said. He encouraged surgeons to work in multidisciplinary teams and reach out to colleagues in the engineering space to help contribute to the development of new tools.
“We need more of you working with the data scientists and engineers.”
Dr. Hashimoto has served as an advisor to Activ Surgical and Johnson & Johnson Institute; received textbook royalties from McGraw Hill Education; has a patent with Mass General Brigham; and served on the board of directors for Global Surgical AI Collaborative.
This article is from the June 2023 print issue.