EDITORIAL| VOLUME 161, ISSUE 2, P303-304, FEBRUARY 01, 2022

The Value of SEP-1

Richard Beale

Chest 2022; 161: 303-304

Twenty years ago, the concept of sepsis as the host response to infection¹ was still largely the obsession of intensivists and infectious diseases specialists. Deteriorating patients were recognized late,² often presenting in a state of collapse, and the lack of a systematic approach to treatment resulted in variable clinical care, with prompt antibiotic administration in particular often being overlooked.³ Fortunately, several developments changed this. More and larger randomized trials were done in the critically ill, finally generating positive results, but also presenting the challenge of implementing the findings. The importance of earlier identification of patients whose condition was deteriorating was recognized, and the sheer scale of the human and financial costs of sepsis⁴ became visible to health systems, payers, and patients and families, resulting in demands for more consistent and evidence-based care.

These factors led directly to the Surviving Sepsis Campaign,⁵the regular publication of evidenced-based sepsis treatment guidelines, and the introduction of sepsis care bundles. A version of the latter methodology formed the basis for both “Rory’s Regulations,” introduced in New York State in 2013, and the SEP-1 measures introduced by the Centers for Medicare and Medicaid Services in 2015, and it is an analysis of the first 18 months of the latter initiative that is presented by Townsend and colleagues⁶ in this issue of CHEST.

The first striking feature of their report is the continuing size of the sepsis burden, with over 1.3 million patients being reported to the SEP-1 program in this period. This allowed 661,457 patients to be matched with Medicare outcome data, generating 333,770 for inclusion in the statistical analysis. Next is the pattern of bundle element compliance. The overall SEP-1 Bundle pass rate was 42.1%, although the individual element pass rate ranged from 90% for taking blood cultures and 88% for giving antibiotics, down to 38.1% for the 6-hour reassessment element for patients in shock. The pass rate for the Severe Sepsis 3- and 6-Hour Bundles were 68.5% and 62.6%, for the Shock 3-Hour Bundle 62.2%, but only 34% for the Shock 6-Hour Bundle. These results demonstrate the challenge of passing an all-or-nothing bundle measure, and how cumulative compliance becomes increasingly difficult in sicker patients, where there is more to do and more opportunity to fail. It is sometimes forgotten that this is a deliberate feature of the bundle methodology, driving organizations to improve their overall systems of care to achieve successful completion. Understanding performance against individual elements is still important to allow specific improvement efforts, although this can be misleading if some elements are more or less likely to be delivered or to influence outcome. Appropriate clinical deviation from bundle elements in individual patients also may be unrecognized, or even seem to be discouraged or penalized. For some clinicians, the resulting perceived loss of professional autonomy is difficult to accept, which also may impact compliance. It is not surprising, therefore, that the sepsis bundle approach and the SEP-1 initiative remain controversial within the clinical community treating patients with sepsis.

To refine their analysis the authors applied sophisticated propensity score matching: by calculating the propensity score of each patient to be SEP-1 compliant and matching compliant and noncompliant patients with similar scores, known covariates are balanced allowing a direct comparison of outcomes to be made. Two differing degrees of stringency were applied, both demonstrating that SEP-1 compliance was associated with reductions in absolute risk of death at the population level (21.8% vs 27.48% mortality; absolute risk reduction, 5.67%; 95% CI, 5.33-6.0 for the ‘standard’ match group and 22.22% vs 26.28% mortality; absolute risk reduction, 4.06%; CI, 3.70-4.41 for the ‘stringent’ match group). The fundamental principle underpinning the validity of this approach is that all the potentially important confounding factors have been measured and allowed for, meaning that the comparison between the patients with a similar propensity score for SEP-1 compliance is a valid one. Given that the overall SEP-1 compliance was quite low, it is reasonable to assume that there was indeed sufficient genuine variation in practice to allow for essentially similar patients being found and matched, in order that a valid comparison could be made. Moreover, the fact that individual element compliance was substantially higher than the overall pass-rate is likely to have narrowed the outcome difference between compliant and non-compliant patients.

When the authors applied a conditional, subject-level analysis, patients in whom the SEP-1 bundles were successfully delivered also had better outcomes, with a reduction in mortality from 30.3%-21.7% (adjusted conditional OR, 0.829; 95% CI, 0.812-0.864; P < .001), although only 10.6% of the whole study population were eligible for either shock bundle, emphasizing that the benefit was largely driven by the effectiveness of the severe sepsis response, and presumably the wider organizational improvements put in place to deliver it. Whilst the authors highlight that vasopressor use within the Shock 6-Hour Bundle was associated with an increased mortality (39.3% vs 29.1%; conditional adjusted OR, 1.317; CI, 1.126-1.541; P < .001), only 5,332 patients were eligible for this treatment.

For those intrinsically distrustful of the SEP-1 program or “big data” analytics, the suspicion may remain that there were other elements at play, visible to clinical eyes, that justified deviating from the bundle but constitute unmeasured confounders undermining the propensity score findings. The authors have addressed this using a sensitivity analysis and suggest that an unmeasured confounder would have to increase the odds of mortality by more than 2 and 1.45 times, respectively, for their “standard” and “stringent” match analyses to undermine their results. Notwithstanding the analytic approach, these are still observational data, so it is important to take a broader clinical perspective on the validity of the results, and their implications for future care. When the Surviving Sepsis Campaign started, sepsis care was poor, with considerable variability. It was entirely plausible that an initiative essentially comprising early recognition, and then delivering the basics of care properly, would have a substantial benefit.⁷ Now, sepsis treatment processes are well established, and indeed performance was relatively stable across the study period. Why, then, should there be such an ongoing failure of compliance? If it is marginal, driven by minor delays and failure because of the binary nature of bundles, or because of appropriate deviation caused by expert clinical judgment, this should not result in the mortality differences seen in the analysis. To discount the results therefore requires that there really are major biases unaccounted for within the data, or that the model and analysis are wrong in some other fundamental way. Alternatively, the analysis is correct, and there still are unaddressed variations in performance within and between institutions, sufficient to generate these mortality differences, despite embedded quality improvement programs. Moreover, although the narrow bundle elements constituting most of the analysis are the interventions for severe sepsis and not for septic shock, there are associated process factors that cannot be measured in an analysis such as this. For instance, how much of the benefit associated with early antibiotic administration comes from the direct immediate benefit of the antibiotics themselves, and how much from all the other factors necessary to deliver appropriate antibiotics promptly with a high degree of reliability,⁸ and their implications for increasing the quality of care in the round?

The authors have carried out the analysis to a very high standard, using state-of-the-art techniques, and it has been reviewed by a statistical expert, but clinical readers will also know themselves whether their own sepsis care is as well organized and consistent as it should be, and therefore how they should respond to these results. To be able to meet targets such as SEP-1, but occasionally to choose to derogate, is a very different thing from simply being unable to achieve these standards. What is clear, however, is that the conversation about how sepsis care should develop must recognize the enormous achievement that is the SEP-1 Program, and the many lives it has very likely saved.