The ‘IDEAL’ Framework for Evaluating New Surgical Techniques
Montreal—New drugs typically undergo rigorous evaluation before being approved for widespread use. In contrast, new surgical techniques often do not endure the same level of scrutiny before gaining traction. In fact, many techniques are adopted without adequate supporting evidence of their efficacy and safety (Lancet 2018;392:88-94).
“Surgeons have long been criticized for the poor quality of their clinical research on new procedures,” said Peter McCulloch, MD, a professor of surgical science and practice in the Nuffield Department of Surgical Sciences, University of Oxford, in England.
The problem, Dr. McCulloch realized, was that the framework for studying new drugs was not outfitted for surgery.
“It became clear that surgeons needed a different pathway to study surgical techniques, one that aligned better with how these techniques evolve naturally,” he said.
That is why in 2009, Dr. McCulloch, along with a team of experts, joined a surgery-specific framework to guide surgeons in their approach to clinical research. This ‘IDEAL’ framework outlines five key stages of developing and studying new surgical procedures: Idea–Development–Exploration–Assessment–Long-term study.
At the 2019 Canadian Surgery Forum, Dr. McCulloch explored how the IDEAL framework has evolved since its inception a decade ago, why the concepts are relevant to surgical research, and how the idea is gaining momentum among surgeons.
Origins and Evolution of IDEAL
During the evidence-based medicine movement in the 1990s, Dr. McCulloch began to explore why surgeons seemed to have difficulties conducting randomized controlled trials (RCTs).
The main challenge, he found, came down to process. When a new drug enters clinical testing, the compound does not change. In fact, “it’s illegal to try to alter a compound at this point,” Dr. McCulloch said.
But the situation in surgery is different because new operations are in a state of flux. “When I’ve asked surgeon innovators whether they perform a new procedure the same way the first and second time, the answer is no,” Dr. McCulloch said. “A new technique is still changing. You have to wait until it is more mature and stable before subjecting it to an RCT.”
In 2007, Dr. McCulloch and a team of experts met at Balliol College in Oxford, England, to develop an approach to clinical trials that aligned better with the realities of how techniques evolve over time. These meetings led to the IDEAL framework, presented in a series of articles in Lancet in 2009 (374:1037).
“We need to make sure we do RCTs in the safest possible way,” Dr. McCulloch said. “If we do trials badly, we end up with wrong answers or no answer.”
The framework is composed of five stages. Stage 1 is the idea or proof-of-concept phase in which surgeons define a new treatment and why it is needed. This stage represents the first time the technique is performed on patients. In stage 2a, the development phase, surgeons refine and optimize the technique until it seems stable; in stage 2b, the exploration phase, surgeons try to reach a more general consensus on the procedure, which includes what variations are acceptable, which patients will benefit most, whether surgeons have adequate experience performing the operation, and whether the technique should be studied in a larger group of patients. Stage 3, the assessment phase, centers on an RCT comparing how the new intervention measures up to the current standard practice. Finally, stage 4 looks at the technique over the long term to pinpoint any adverse effects or changes in quality. At this point, data are collected in a registry.
“A lot of this framework is common sense,” Dr. McCulloch said. “But the process in surgery has largely been unregulated and unstructured, and the IDEAL framework provides a formalized approach.”
Dr. McCulloch and his colleagues recently expanded these IDEAL concepts to address the introduction of new technologies and devices as well as highlight ethical issues that surgeons should consider at each stage (Ann Surg 2019;269:211-220). The investigators, for example, incorporated a preclinical step that involves working with animal models, simulators and cadavers, and proposed introducing registries at stage 1 instead of stage 4 because “starting a registry at an early stage can allow data harmonization and pooling of resources,” they wrote.
The main ethical goals are improving patient safety and minimizing harm, which involves transparent reporting of outcomes, patient informed consent, and ongoing assessments of surgical performance and learning curves.
“Transparency is key to this whole endeavor—what you did, why you did it and what happened,” Dr. McCulloch said. “We recommend reporting all cases and tweaks to the technique until surgeons are satisfied with the operation and no longer have plans to change it.”
IDEAL in Practice
When Carl Brown, MD, FRCSC, saw the 2009 Lancet papers on the IDEAL framework, he reached out to Dr. McCulloch for more details.
“I’ve always had a strong interest in the ethical adoption of new techniques and technology, and was intrigued by the IDEAL concepts,” said Dr. Brown, the head of the Division of General Surgery at St. Paul’s Hospital and chair of the Section of Colorectal Surgery at the University of British Columbia, in Vancouver.
Dr. Brown and his colleagues had a particular technique in mind: a new minimally invasive approach to rectal cancer, known as transanal total mesorectal excision (TaTME). The approach could potentially advance rectal cancer surgery, but posed technical challenges and required careful study to ensure surgeons performed the dissection in the correct plane. Dr. Brown’s team used the IDEAL criteria to evaluate TaTME and implement it safely.
“We met as a group to create clear stipulations about how and when the technique should be offered to patients and how to consent patients,” Dr. Brown said.
Before surgery, Dr. Brown and his colleagues made patients aware that TaTME was still experimental and what the potential risks were. In the OR, two surgeons worked on the transanal component—one surgeon performed it while the other surgeon assisted and provided feedback.
“This collaboration helped enhance successful outcomes, and increased the experience of surgeons who benefited from watching and assisting,” Dr. Brown said.
Next, they created an international TaTME registry involving 23 countries to fuel this collaboration. The registry contains outcomes for more than 1,500 patients, enabling surgeons to assess the safety of the procedure and pinpoint problem areas (Ann Surg 2019;269:700-711).
The story of TaTME also highlights the importance of carefully evaluating a new technique before disseminating it more widely. In December 2018, the Norwegian Colorectal Cancer Group recommended a temporary national moratorium on TaTME after a study found a local recurrence rate of 9.5% compared with 3.4% after laparoscopic TME (Br J Surg 2019;106:1120-1121).
“Norway stopped surgeons from doing this procedure because of the higher recurrence of cancer,” said Dr. Brown, who cautioned that the technique may have been introduced at some institutions without sufficient oversight. Surgeons writing on behalf of the Norwegian Colorectal Cancer Group agreed, explaining that the learning curve seems to be “the Achilles heel of surgical innovation,” citing the IDEAL framework as a model for how to ethically introduce new techniques (Br J Surg 2019;106:1120-1121).
IDEAL is also being adopted in urology, with an increasing number of publications citing the framework (BJU Int 2019;123:1078-1085). The U.K. Royal College of Surgeons and Annals of Surgery recently endorsed it, too.
Despite gaining recognition, the IDEAL framework has not yet been adopted widely in practice. Surgeons may have reservations that journals and funders will not be on board or that too much transparency will put surgeons at risk for litigation.
“Change takes time, but I’m encouraged we will see a shift in how clinical trials are performed,” Dr. McCulloch said. “People may have concerns about publishing papers or getting grant funding using this framework, but I hope that with the two recent endorsements and with more people embracing it, there will be momentum for wider adoption.”