img-button

During this coronavirus pandemic, I have been on lockdown at home because I’m considered a senior citizen while many of you have been fighting bravely on the front lines. I have had a lot of time to watch the never-ending parade of news anchors speculate on what we’ve done right and what we’ve done wrong. On television, I have witnessed the tragedy of this pandemic, both in lives lost and the financial suffering. I have participated in many educational Zooms (not the noise a child makes when playing with cars but instead the modern way we can communicate and still socially distance).

img-button

I and many others have asked the question: Could we, and should we, have done better in handling this pandemic? Why have our medical system and government been so confused on the path to take during this viral pandemic? I have also asked myself: Will we learn anything during the tragedy that will prepare us for future equally tragic events? Who are the experts we should be relying on now and in the future? Finally, as a scientist, I ask: Is there a better way that we should be collecting the data that should be the basis for making decisions in the future?

We've asked two physicians to weigh in on the problem. We have asked them: What is the first thing we need to do to fix our health care system? As you will see, they have slightly different takes on the situation; both seem to say the system is broken but there is a way to fix it. You will see from their contributions that both feel our health care system is similar to Babylon in that there has been a breakdown in communication.

ADVERTISEMENT

I hope this month’s debate will make you think of ways we can improve the system. This is not a question for our politicians to answer, but rather for you, the scientists and the ones on the front lines, who are so deeply involved in health care. We cannot leave it to others to solve the problem, or the next pandemic may be the last.

Edward L. Felix, MD
Editor, The Great Debates
General Surgeon, Pismo Beach, Calif.

The First Thing We Need to Do Is Create an Integrated Electronic Health Record

David Earle, MD
Assoc. Professor of Surgery, Tufts University School of Medicine
Director, New England Hernia Center
North Chelmsford, Mass.
img-button

The goal of health care reform is to improve value by increasing quality and decreasing cost. The Affordable Care Act, passed in 2010, accomplished the opposite by focusing on reducing cost and increasing volume. To accomplish our goal, we first need an integrated electronic health record (EHR).

From chaos can come order. In that regard, my hope is that COVID-19 becomes the “Tenerife” of health care.

Tenerife is a small Spanish island in the Canary Islands archipelago off the coast of West Africa. There, on March 27, 1977, two 747 passenger jets collided, resulting in the deadliest aviation accident in history—583 dead, 61 survivors. When KLM Airlines got news of the accident, they looked for their best pilot, Capt. Jacob van Zanten, to investigate the accident. They soon found that their best pilot had hit Pan Am Flight 1736 on the runway during takeoff, and was killed instantly.

ADVERTISEMENT

On Jan. 31, 2020, the U.S. government and multiple airlines banned travel to the United States from China due to a novel coronavirus. Many subsequent actions were taken to “flatten the curve” of rising COVID-19 to prevent overwhelming our health care system, including shutting down the economic engine of hospitals—the operating room. As the pandemic spread, elective surgeries and census numbers plummeted, and hospitals responded by dropping hundreds of employees and dozens of services. Modern data analytics drove us to these decisions. The problem isn’t the analytics, it’s the data. They are incomplete, and maybe inaccurate.

As models emerged from the Gates Foundation–funded Institute for Health Metrics and Evaluation (IHME), the public lockdown began a metamorphosis, even as the models fluctuated wildly, and were not specific. On May 6, the IHME estimated that 43,511 to 91,454 hospital beds were needed to deal with the pandemic.1 That information, however, is worthless for hospital officials because no action can be taken. But that doesn’t stop elected officials from using it to make decisions. Garbage in, garbage out. The best data analytics can’t overcome incomplete or incorrect data.

For a striking example of modern data analysis with poor-quality data, consider the latest robotic systems used to help pilots fly planes. After all, machines can do it better, right? Look at all the marketing for surgical robotics to corroborate that philosophy. In an effort to keep up with rival Airbus, Boeing released its most modern aircraft, the 737 MAX. The larger, more efficient engines could cause the plane’s nose to tilt upward, so data analytics software was used to compensate for this. However, between Oct. 29, 2018 and March 10, 2019, when sensor information was erroneous, incomplete and/or out of context, the software still corrected the plane’s altitude and caused two brand-new aircraft, loaded with a total of 346 passengers and crew, to crash shortly after takeoff, killing all aboard.2 Garbage in, garbage out can kill. God rest their souls.

ADVERTISEMENT

So we do need data analytics, but we can’t do it without the data. The CDC has known for decades the importance of collecting good data, so it created the National Healthcare Safety Network (NHSN) database, a de facto EHR. This single database has modules for diseases, including COVID-19, but it has fatal flaws: 1) it only captures a very limited number of diseases, and 2) it primarily gets data from hospitals and dialysis centers.3 There are no data from outpatient settings, no pharmacy data, no community health clinic data. Also, en route to the CDC, the data are passed along like a story whispered around a campfire, propagating errors along the way. Yes, another incomplete data set, setting public policy.

We then heard about promising treatment options for COVID-19, and how regulatory restrictions would be lowered to speed their legal marketing. Anthony Fauci, MD, of the Presidential Coronavirus Task Force, then stated we must use randomized controlled trials to figure out whether any treatments are effective. And RCTs take time—years. Time is what we don’t have during a pandemic.

Consider the number of unique drug combinations using four medications: It’s 24. Then consider timing and dosing variations. A power analysis would reveal that tens of thousands of highly selected patients would be required, which would take years. And that statistical difference may not be clinically relevant. Additionally, RCTs have exclusion criteria, are difficult to interpret and necessarily incomplete representations of the actual data,4 which limits their applicability in the real world where we all work. That is precisely why an RCT will not answer the question about which treatment is best for COVID-19, just as the soup maker Campbell’s discovered there is no best spaghetti sauce in the 1980s.5 This complexity is perhaps one reason the Federal Aviation Administration came out with its “Call to Action” in August 2007, to address aviation runway incursions, with the Tenerife accident playing a prominent role in the decision to proceed with this project. I’ll do the math for you: It’s 30 years after the accident.

ADVERTISEMENT

The solution starts with accurate, timely data. We already collect it in our EHRs, but the data aren’t integrated and the corporate masters won’t give us access. If we had access to accurate and complete data, we subsequently analyze thousands of real-world patient outcomes for COVID-19 treatments to figure out which works best for an individual at a particular location—not a group of patients spread out in different environments. For this, we need an integrated EHR system. With accurate data, we could manage COVID-19 (and all diseases) with a scalpel instead of a sledgehammer.

The only thing worse than no data is incomplete or inaccurate data. But the data become dramatically more flawed as assumptions replace real-world data, and we all know what happens when you assume.6

References

  1. Institute for Health Metrics and Evaluation. https://covid19.healthdata.org/ united-states-of-america. Accessed May 6, 2020.
  2. House Committee on Transportation and Infrastructure. The Boeing 737 MAX Aircraft: Costs, Consequences, and Lessons from its Design, Development, and Certification—Preliminary Investigative Findings, March 2020. https://bit.ly/ 3bS2CsT
  3. CDC. www.cdc.gov/ nhsn/ about-nhsn/ index.html. Accessed May 8, 2020.
  4. Bauchner H, Golub RM, Fontanarosa PB. JAMA. 2019;322(8):732-735. doi:10.1001/jama.2019.12056
  5. Malcolm Gladwell. www.ted.com/ talks/ malcolm_gladwell_choice_happiness_and_spaghetti_sauce?language=en#t-313902
  6. American Broadcasting Company. “The Odd Couple.” 2011. https://youtu.be/ svkgOsr7pUc
ADVERTISEMENT

The First Thing We Need to Do Is Develop a Data Analytics Infrastructure Based on Systems and Data Science

Bruce Ramshaw, MD
General Surgeon
Managing Partner, CQInsights
Knoxville, Tenn.
img-button

In 1944, President Franklin D. Roosevelt sent a letter requesting Vannevar Bush, the director of the Office of Scientific Research and Development, to propose a plan for applying similar research principles used in the recent war (World War II) and apply them to the war on disease. The Bush report, “Science—The Endless Frontier,” led to the development of the National Science Foundation and then the National Institutes of Health. The model that was applied is termed the “linear model of innovation,” and its focus is on basic science using reductionist tools, such as controlled research trials.

One example of this reductionist model for innovation is the Norden bombsight. An engineer, Carl Norden, wanted to build a bombsight that improved accuracy to allow bombers to fly at a safer height. He developed a complicated machine, with about 2,000 parts, based on his expertise in gyro stabilization. In controlled testing, the bombsight performed brilliantly. The CEP (a circle into which 50% of bombs would fall) was 75 feet, a fantastic performance for that time. Without testing in the real world, the Norden bombsight was mass produced, and tens of thousands were installed in planes for the U.S. Navy and Air Force at a cost of nearly $10,000 each.

When the bombsight was eventually used in real-world conditions, the CEP increased to 1,200 feet, a decay in accuracy of over 90%—about the level of decay in accuracy for our current airport screening process in real-world conditions versus controlled testing. The controlled environment did not account for several factors that might affect accuracy, such as weather conditions, mechanical issues, and the fact that the person operating the bombsight might be getting shot at. The performance was so bad that its usage was abandoned soon after it was introduced into combat.

A different model for innovation comes from the principles of systems and data science in real-world conditions, using analytics and feedback loops providing iterative improvement. One example of this is the Wright brothers’ successful achievement of flight. They used real-world conditions, collecting and analyzing data with feedback loops to improve their designs. At one point, they tested dozens of wing surfaces over two months, analyzing the data as results were generated. After their success, one of the Wright brothers wrote: “it is doubtful if anyone would have ever developed a flyable wing without first developing this data.”

If we continue to use reductionist tools for the COVID-19 pandemic, we will continue to require the same management strategies we applied almost 400 years ago during the plague—shelter in place, social distancing, quarantines, etc. And our linear model for innovation, waiting for the results of controlled clinical trials, will result in lengthy delays and unintended waste and harm when treatments are introduced into real-world patient care.

Controlled studies attempt to determine whether a treatment works or doesn’t work, and generates recommendations for the average patient. This one-size-fits-all approach is not appropriate for a complex biological system. With an appropriate data analytics infrastructure in place, we could identify different patient subpopulations and apply the optimal variety of treatments matched appropriately to these subpopulations. This is the type of analytics that Netflix uses to present the optimal variety of movies and shows to different subpopulations of people—and thank goodness for Netflix and its algorithms during this pandemic.

Many treatments are already being used in clinical settings to improve outcomes for patients with COVID-19, but without a data infrastructure in place that can measure outcomes and assess the value of any treatment in different subpopulations, we are flying blind. We could also use a data infrastructure to identify the factors that lead to a subpopulation that gets the virus and has minimal or no symptoms, one that has moderate symptoms, and another that develops a severe illness and is at risk for dying. The best strategy we have now in the real world is doctors and nurses posting their observations on social media and other forms of communication. But without a data and analytics infrastructure, these stories are just anecdotes.

A real-world data infrastructure should include input from the front-line clinical team in each clinical environment. This system would enable a human-computing symbiosis in which the clinical team could identify the most important patient and treatment factors that would influence outcomes that measure the value of care (see “Value Over Volume,” General Surgery News March 2020). The data set could then be uploaded into analytics and visualization tools, and the clinical team could then interpret the results and apply insights to improve measurements and generate ideas to improve outcomes. Each clinical team could be networked with other teams to share their learning and improve algorithms to identify subpopulations for an optimal variety of treatment and preventative measures, termed the “ensemble model for learning.”

Our health care system has spent billions of dollars for a data infrastructure designed for documentation, coding and billing. We have fragmented EHRs, hundreds of revenue cycle management software products, and armies of coders and billers, guided by the system, in every clinical environment, all focused on getting paid. If we applied a fraction of those resources to a data infrastructure to support the measurement and improvement of value-based outcomes for real-world patient care management, we would have a sustainable health care system globally, and this pandemic would likely be managed and resolved by now. I believe this type of data infrastructure is the most important lesson to learn and apply from the tragedy of this pandemic.

Visit Dr. Ramshaw’s blog at www.bruceramshaw.com/ blog.


Dr. Earle’s Rebuttal to Dr. Ramshaw

Bruce Ramshaw has been speaking about complex systems management for at least a decade. I largely agree with what he has to say, and have realized that literally all aspects of medicine and health care delivery are somehow not using real-world data to continuously improve. In Dr. Ramshaw’s own words, “With an appropriate data analytics infrastructure in place, we could identify … and apply the optimal variety of treatments matched appropriately to these subpopulations.” I agree. But without data, the data analytics infrastructure will be in the next season of television’s “Mysteries of the Abandoned.”

Dr. Ramshaw realizes this, stating: “Many treatments are already being used in clinical settings to improve outcomes for patients with COVID-19, but without a data infrastructure in place …, we are flying blind.” Once again, no data means no data analysis.

He does acknowledge that “[a] real-world data infrastructure should include input from the front-line clinical team in each clinical environment” and “the best we have now in the real world is doctors and nurses posting their observations on social media and other forms of communication.” As I stated in my argument, we already collect data from front-line clinical teams, we just don’t have access to it. Neither do the elected officials and neither does the CDC. Dr. Ramshaw pointed out the Wright brothers’ statement: “it is doubtful if anyone would have ever developed a flyable wing without first developing this data.” Even the Wright brothers recognized the need for accurate and complete data, before analyzing and acting on it.

So, the data we need are collected in software paid for by doctors and hospitals, but doctors and hospitals don’t have access. The FDA doesn’t have access. The CDC doesn’t have access. The multibillion-dollar software companies are hoarding the data. Why? We can only guess, but it’s not to improve health care. This is the true role of government—legislate an integrated EHR [electronic health record]! Give us access to the data we desperately need to apply Dr. Ramshaw’s data analytics infrastructure, protected by all the laws on the books today, such as HIPAA.

Our lives literally depend on it.


Dr. Ramshaw’s Rebuttal to Dr. Earle

I appreciate the argument presented by Dr. Earle for an integrated EHR. It seems like we should be able to share data across our health care system. But we already have an integrated EHR in the United States, in the Veterans Affairs (VA) system. It is convenient for clinicians and patients to access data throughout that system. But that isn’t how data science works (if it is used to measure and improve outcomes). To achieve value from data and software, a system should be decentralized into each context (definable patient care process) and available in each local clinical environment. A decentralized data system is also the optimal solution for data security. Centralized data is prone to major accidental and intentional data breaches.

A couple of years ago, the Veterans Administration announced it would be moving from the current EHR to a new one from Cerner that would cost $10 billion. A couple of months ago, the agency announced delays and the cost is now estimated to be $16.1 billion. Unfortunately, no centralized data repository will ever drive better value into the system (no matter how much we spend). Data access is not the most important solution. Understanding data science and applying it in the real world of patient care is the most important solution (and would be much less costly). This would not be the first decentralized global data infrastructure in our world built on principles of data science; that’s how the internet was designed and built at a total cost estimate of about $125 million.

Recently, during a Senate hearing about the lessons learned from our response to COVID-19, Sen. Mitt Romney (R-Utah) asked Dr. Robert Redfield, the director of the CDC: “How is it possible in this day and age that the CDC has never established a real-time system with accurate data?” Dr. Redfield responded, “The reality is there is an archaic system. … This nation needs a modern, highly capable data analytic system that can do predictive analysis. I think it’s one of the many shortcomings that have been identified as we went through this outbreak, and I couldn’t agree with you more, it’s time to get that corrected.” I agree, as well.