Quality Group Rejects Appeal on Sepsis Measure

The National Quality Forum (NQF) has rejected an appeal of its decision to endorse a measure for treating sepsis, leaving emergency physicians and other critics of the measure disappointed, while the measure developers are praising the outcome.

The quality measure, known as the Severe Sepsis and Septic Shock: Management Bundle, or SEP-1, was first endorsed in 2013 by the NQF, and first implemented by the Centers for Medicare & Medicaid Services (CMS) in October 2015. In addition to being several dozen pages long, the measure has another unusual feature — rather than being stewarded by the American Medical Association, CMS, or an infectious diseases association, its steward is Henry Ford Hospital in Detroit.

Controversy Over the Measure

Although the measure is long, it has seven basic elements, according to Sean Townsend, MD, vice president of quality and safety at California Pacific Medical Center in San Francisco, and one of the measure’s co-developers:

• Measure the patient’s initial lactate level
• Take a blood culture before administering a broad-spectrum antibiotic
• Administer a broad-spectrum antibiotic within 3 hours
• Administer 30 mL/kg of crystalloid fluid to patients with low blood pressure or elevated lactate
• Repeat the lactate test if it was elevated initially
• Administer a vasopressor for patients who remain hypotensive
• Reassess the patient after a certain period

The measure has engendered a fair amount of controversy, explained Michael Klompas, MD, MPH, and Chanu Rhee, MD, MPH, both of Harvard Medical School in Boston, in a 2021 Annals of Internal Medicine editorial.

“Some believe it has brought welcome attention to sepsis and catalyzed hospitals to implement protocols that save lives,” they wrote. “Others are concerned that the measure’s rigid treatment requirements and aggressive timelines encourage excessive fluids and antibiotics, curtail clinicians’ latitude to calibrate their management to their certainty of infection and patients’ severity of illness, and unduly anchor providers’ diagnostic focus on sepsis to the exclusion of other serious conditions.”

The editorial came in response to an Annals study by Ian Barbash, MD, of the University of Pittsburgh, and colleagues that found that the sepsis measure, when implemented at one healthcare system, was associated with “a 50% increase in lactate checks (70% observed vs 47% expected if pre-SEP-1 trends had continued), a 10% increase in broad-spectrum antibiotic administration (50% vs 45%), and a 30% increase in infusions of 30 mL/kg of intravenous fluid (13% vs 10%) within 3 hours of culture orders,” wrote Klompas and Rhee.

Appeal Filed in January

In 2021, the NQF re-endorsed the measure for a third time. Several infectious disease and emergency medicine groups, including the Infectious Diseases Society of America (IDSA), the American College of Emergency Physicians (ACEP), and the Society of Hospital Medicine, filed an appeal with NQF in January 2022, expressing concerns about the way the re-endorsement process was handled.

At a virtual appeals board hearing on April 29, Klompas, who spoke for the appellants, focused on the fact that two members of the appeals board were recused from participating due to conflicts of interest. “The apparent conflicts of interest, however, were nothing more than a deep interest in sepsis and having a record of having published technical critiques of SEP-1, including recommendations for possible modifications,” he said. “These are not conflicts in our opinion, but rather bona fides that demonstrate these members’ high-level interest and expertise in sepsis in SEP-1.”

The exclusion of these key members for these reasons “was particularly disturbing, because the measure stewards — who arguably have identical conflicts by virtue of their investment in the measure — were allowed to participate and, by all accounts, did so vigorously, including repeatedly interrupting and contradicting speakers, restraining the realm of conversation,” he added.

Townsend, who spoke for the measure developers, had a different view. The recused board members “published articles critical of the sufficiency of the evidence supporting NQF 0500,” he said. “One of the measure evaluation criteria is the sufficiency of the evidence. These opinions conflicted with their duties to serve as impartial judges. And they could not render an impartial decision.” In addition, said Townsend, “both committee members engaged in work with the measure developers. The NQF policy requires that if committee members have done so, that they must be recused.”

Townsend also argued that although the measure discussion was spirited, “there are no published rules of orders outlining parliamentary procedures at NQF meetings. I wish there were. If there were no published rules, all the lapses that may have occurred are subjective.” For example, he said, “if there’s no means to signal you’d like to be heard, what should you do? Is interjecting to counter misinformation a lapse of decorum?”

On the appellant side, Klompas also argued that “important new data that has emerged since SEP-1 was last evaluated by the [NQF’s] Patient Safety Committee.” He cited three studies, including the Annals study, noting that “all three studies documented that, in real-world practice, SEP-1 is associated with significant increases in … lactate-checking and broad-spectrum antibiotic utilization, but no change in meaningful patient outcomes.”

“Depending on the study, there was either no change in hospital mortality … no change in the combined outcome of hospital death or discharge to hospice … or in fact, in the study coordinated by Duke University, there was an increase in the relative mortality trend following the implementation of SEP-1,” Klompas said. “We have a shared goal of improving sepsis care and outcomes. And the consensus of six professional societies is that SEP-1 has not achieved this goal. It is therefore imperative for us to modify this measure if we are to realize our shared goal of saving more lives.”

Townsend argued that the studies should not be included in the appeals board’s decision making because they weren’t mentioned in the group’s letter to the appeals board. The studies “are therefore probably not appropriately discussed in this forum because the contents of the appeal letter should be the topic of this discussion,” he said.

Unanimous Vote to Re-Endorse

In the end, the five participating members of the NQF’s appeals board voted against the appeal, and the vote was unanimous (5-0) on the questions presented: the board agreed there were no procedural errors reasonably likely to affect the outcome of the original endorsement decision, and that there was no new evidence presented that would have affected the original endorsement.

ACEP expressed disappointment with the outcome. “As emergency physicians, we strive to provide the highest quality care to patients with sepsis and all manner of other illness or injury,” Aisha Terry, MD, MPH, an emergency physician in Washington, D.C. and a member of ACEP’s board of directors, said in a statement. “While we are disappointed that NQF rejected the appeal of ACEP and other professional societies, including those representing infectious disease, we will continue to support research and gather data to ensure that all patients receive the best possible care.”

On the developer side, “We are pleased that the NQF Appeals Board honored the work of the NQF approved experts in the patient safety committee who approved the measure prior to the organizations’ appeal,” Townsend wrote in an email. “We believe the NQF process works because unbiased experts in the field vote with their conscience without undue external pressures.”

He added that the measure developers had been working with some of the appellant organizations prior to the appeal being filed. “We are disappointed they broke their agreement not to oppose the measure while they were working to improve it with us,” Townsend wrote. “Because the appellants reversed their commitment to us, we suspended work with them and advised them that this was the reason we could not work together. Under the proper conditions, this work could resume, but the appellants would have to honor their commitments with a memorandum of understanding.”