Scoring systems for the characterization of sepsis and associated outcomes

Background

Sepsis is of great clinical importance, being responsible for more than a third of all hospital admissions and approximately 50% of all intensive care unit (ICU) admissions (1). It is associated with a large economic burden on healthcare resources which is likely to worsen given the apparent increase in the incidence of sepsis (1,2).

Sepsis is associated with mortality of up to 40% and approximately a third of non-survivors die within the first 48 hours of admission to ICU (1). It is widely asserted that early diagnosis and the prompt initiation of treatment, especially antimicrobials and fluid resuscitation are associated with a better outcome (3-5). One of the main barriers to early interventions in sepsis is the lack of diagnostic tools and this is compounded by the fact that sepsis is a heterogeneous and enigmatic syndrome with no gold standard for diagnosis. For many years, the systemic inflammatory response syndrome (SIRS) criteria were considered to be central to the diagnosis, promoting the importance of inflammation. However, whilst SIRS is clearly associated with mortality, it both overly sensitive (6) and at the same time yields up to 1 in 8 false negatives in patients with infection and organ failure (7).

Sepsis definitions

In this context, the authors of The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) (8) reviewed and updated sepsis definitions, benefiting from the advances in understanding of the pathobiology which have occurred since the last revision in 2001 (9). Sepsis is now defined as a “life-threatening organ dysfunction caused by a dysregulated host response to infection”. The authors defined organ dysfunction as an increase in the Sequential (Sepsis-related) Organ Failure Assessment (SOFA) score or ≥2, and this was associated with a 10% mortality risk (10). However, because of a lack of familiarity with SOFA outside of the ICU, and because SOFA requires laboratory values which may not be rapidly available, the Quick SOFA (qSOFA) was developed to provide an abbreviated version that can easily be performed at the bedside by the non-specialist (10). It was developed using a parsimonious model to achieve a simple scoring system with the fewest number of variables associated with the greatest predictive ability. The main utility of qSOFA appears to be for the characterisation of patients with suspected or known infection, in whom sepsis should be considered, who are at a higher risk of developing a poor outcome, and who may benefit from more frequent observations and targeted interventions (i.e., sepsis bundle and or Critical Care admission). In this context it is acting as a risk predictor although it is not specifically part of the diagnosis of sepsis (11). At the same time qSOFA may act as a surveillance tool and it is suggested that in patients not yet recognised to have infection, a positive qSOFA could also be of value to prompt the consideration of infection. A vital role of Sepsis-3 is to help standardise diagnostic criteria for the purposes of future research studies.

One of the main strengths of the Sepsis-3 definition is that it moves away from the centrality of inflammation and the SIRS criteria, which may simply be an adaptive response and which may be synonymous with infection (8). Instead, by focusing on organ failure they encapsulate the fact that sepsis is both dysregulated and a maladaptive response. Sepsis-3 benefits from being data driven, simple to use and at the same time more specific than SIRS based criteria.

Despite these strengths there are potential concerns with the new definitions. The threshold of SOFA ≥2 identifies patients with a mortality risk of 10% and it is unclear why this threshold was used. Whilst this ensures a minimum level of severity for the definition, in terms of potential use as a surveillance tool this is arguably too high. In a hypothetical patient scenario, with occult sepsis leading to progressive physiological derangement, detection by qSOFA and initiation of interventions may not occur until relatively late in the process. The Sepsis-3 authors recognised that theirs was unlikely to be a definitive definition and that future iterations may need to go beyond SOFA. Indeed, they encouraged the validation of the definition in other settings. Finally, whilst not a criticism specific to Sepsis-3, any significantly new definition does pose the risk of confusion and has the potential to disrupt established quality improvement programs if it does not go hand in hand with meticulous education and operationalisation.

Early warning scores (EWS)

EWS developed in the 1990s in response to a recognition that a significant proportion of patients who suffered adverse events such as cardiac arrest or unplanned admission to Critical Care had deranged physiological parameters for many hours before the event (12). It was suggested that if regular physiological observations were linked to thresholds or triggers for seeking assistance, coupled with the development of rapid response teams (RRT) then deterioration may be able to be prevented. In the last decade, the use of EWS has been widely recommended (13,14) to be implemented in acute hospitals and a proliferation of systems occurred. EWS may be classified as single parameter criteria, multiple parameters and aggregate weighted systems, the latter of which have become most common in the UK (15). Physiological parameters (e.g., heart rate, systolic blood pressure) are measured and a numerical weighting applied according to the degree of deviation from a “normal range”. They are designed for repeated measurement to track patients during their pathway of illness and to trigger at predefined thresholds of aggregate scores. Typically, there is a graded response with intermediate scores demanding a local response from ward based clinicians and higher scores triggering the involvement of RRTs. Because of the profusion of different systems—the UK led the way in agreeing a nationwide standardised system the National Early Warning Score (NEWS), launched in 2012 (16).

Whilst EWS have been widely adopted and whilst their use appears intuitive, there is relatively limited evidence that they improve patient outcome (12,17). Most EWS, including the NEWS were developed by expert opinion and the NEWS was originally only validated in a single center UK hospital cohort. Concerns have been expressed about the NEWS which includes very high sensitivity when compared to previous EWS, the exclusion of some key parameters including urine output and the fact that it is rather general and may not cope well with specific patient populations or chronically abnormal physiology. Nevertheless, EWS and the NEWS in particular, have been considered an important advance in the care of acutely ill patients.

Comparing qSOFA with conventional EWS

In the context of an array of EWS and evolving definitions for sepsis, Churpek and colleagues conducted a study to compare the accuracy of qSOFA to SIRS, the Modified Early Warning Score (MEWS), and NEWS, for the prediction of death or unplanned ICU admission. They studied a population of 30,677 patients with suspected infection (defined as having at least one culture order and at least one order for intravenous antibiotics) outside of the ICU. Of these patients, 60% were from the Emergency Department and 40% from wards; the mortality rate was 5.4% and 23.2% were admitted to the ICU after meeting infection criteria. At the time of infection identification, 51% had ≥2 SIRS compared with just 9% ≥2 qSOFA. In this study, the NEWS outperformed all the other scores, with qSOFA rated third, superior only to SIRS. Using the commonly used thresholds (≥7 NEWS, ≥5 MEWS, ≥2 qSOFA and ≥2 SIRS), SIRS was associated with the highest sensitivity but a very low specificity, whereas NEWS was associated with the second highest sensitivity and a moderate level of specificity. qSOFA had the lowest sensitivity but the second highest specificity.

The cumulative percentage of patients meeting the NEWS, SIRS and qSOFA thresholds in the 48 hours prior to the composite outcome was used to illustrate the length of time between reaching the threshold and the outcome. This demonstrated that qSOFA criteria were only met 5 hours before death or ICU admission, some 12 hours after the SIRS criteria were met and 7 hours after NEWS.

To the best of our knowledge this is the first study which has aimed to validate qSOFA against other EWS and the results seem clear. Nevertheless, there are limitations. This study was conducted in a single center, academic setting in the US and hence the results may not necessarily be generalizable, particularly as thresholds for admission to ICUs varies considerably between and within countries (18). In addition, the validation dataset spans 8 years over which time there may have been changes in practice such as thresholds for Critical Care admission or use of sepsis bundles and there is limited information on the pre-existing use of EWS and RRTs in this institution. Finally, the imputation of median data may have acted to reduce the apparent sensitivity of all EWS.

Discussion

Fundamental to optimising the detection of abnormal physiology associated with sepsis, is a consideration of the illness trajectory. Sepsis is very complex with multiple levels of heterogeneity. Factors affecting an individual’s disease course can be divided into their predisposition to an adverse outcome, the aetiology and severity of the infection itself and the response of the host to the infection (19). These factors will affect the patient’s physiological response to sepsis and therefore for different individuals, the criteria for sepsis may be met at quite different stages of their disease course. On the other hand, for a single individual with sepsis, it is likely that (in the absence of medical intervention) they will follow an approximately linear trajectory of physiological deterioration from mild derangement to severe abnormalities prior to eventual death. It is interesting to consider whether early intervention (perhaps prompted by a high sensitivity EWS trigger) and early initiation of a sepsis bundle could alter the illness trajectory for responders and prevent them from ever meeting more stringent criteria. In this scenario, a lower sensitivity score may be more likely to detect a group of patients who are defined by non-response to initial therapy.

The study by Churpek and colleagues (20) demonstrates that conventional track and trigger EWS may detect sepsis patients with adverse outcomes earlier. Why might this be the case? It may be argued that this relates to the number of variables being monitored. It is intuitive that the greater the number of measures undertaken, the more likely you are to detect an abnormality. However, in our opinion this may not be a complete explanation. Indeed, because there is likely to be coupling between parameters, an abnormality in one is commonly associated with an abnormality in one or more of the others and therefore some parameters may in effect be redundant. This is especially relevant when parameters pertain to the same organ dysfunction i.e., respiratory rate coupled to low oxygen saturations coupled to oxygen administration. An additional explanation may relate to the thresholds. For example, NEWS starts to score for systolic blood pressure at <111 mmHg so detecting earlier degrees of hypotension and respiratory rate starts to score at 21/min rather than the qSOFA value of ≥22/min. Finally, the EWS scores with physiological variance in both directions and it is conceivable that the ability to detect a low respiratory rate could be relevant to some patients who are obtunded by sepsis.

What are the implications of a higher sensitivity EWS and what is an appropriate level of sensitivity?

Using a high sensitivity trigger is likely to lead to earlier identification of patients and the prompter mobilisation of an RRT, with the potential to provide early treatment and hence improve outcome. However, this must be balanced against the risks of trigger fatigue, over work and distraction. Furthermore, there may be a danger of over treatment if a sepsis bundle is administered to patients who either do not have sepsis at all, or who have sepsis but with a very low risk of death and there are legitimate concerns associated with excessive fluid administration (21) or overuse of antibiotics.

Sepsis quality improvement programs to date (i.e., Surviving Sepsis Campaign) have been predicated on the previous sepsis definitions and have been associated with improvements in patient outcome (5). Whilst improvements in sepsis definitions are welcome it is not explicit that applying sepsis bundle interventions at a potentially later stage in the patients’ illness will yield the same outcomes and this may require prospective testing.

The diagnosis and management of sepsis remains very challenging and one size may not fit all. It is not clear from the Sepsis-3 publications that the authors ever intended for qSOFA to rival the early warning systems that have become an integral part of the monitoring of hospital inpatients and it is not in itself a track and trigger system. The Sepsis-3 authors acknowledge that there may remain a role for SIRS criteria in the general identification of infection and indeed in Sepsis 3, SIRS had a similar discrimination to SOFA for patients outside of the ICU.

Whilst high specificity criteria carrying clear associations with mortality may be appropriate for the necessary advancement of sepsis research, for the clinical goal of early detection, a high sensitivity surveillance tool may be more appropriate. The two are not mutually exclusive and may even be complementary. A NEWS type system could be used across a healthcare institution, providing a high sensitivity trigger to alert an RRT who have specific skills in identification and treatment of sepsis and who subsequently use more refined tools (i.e., SOFA) to guide subsequent management decisions. Ultimately, as the authors of both Sepsis-3 and NEWS are at pains to point out, experienced clinical judgement must always be pre-eminent. It is recognised that no scoring system can represent a stand-alone definition of sepsis and neither should the absence of qSOFA, SIRS or EWS criteria prevent a clinician from engaging in the prompt investigation and management of a patient with suspected sepsis.

Acknowledgements

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Blanco J, Muriel-Bombín A, Sagredo V, et al. Incidence, organ dysfunction and mortality in severe sepsis: a Spanish multicentre study. Crit Care 2008;12:R158. [Crossref] [PubMed]
2. Harrison DA, Welch CA, Eddleston JM. The epidemiology of severe sepsis in England, Wales and Northern Ireland, 1996 to 2004: secondary analysis of a high quality clinical database, the ICNARC Case Mix Programme Database. Crit Care 2006;10:R42. [Crossref] [PubMed]
3. Rivers E, Nguyen B, Havstad S, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med 2001;345:1368-77. [Crossref] [PubMed]
4. Kumar A, Roberts D, Wood KE, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006;34:1589-96. [Crossref] [PubMed]
5. Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Intensive Care Med 2010;36:222-31. [Crossref] [PubMed]
6. Churpek MM, Zadravecz FJ, Winslow C, et al. Incidence and prognostic value of the systemic inflammatory response syndrome and organ dysfunctions in ward patients. Am J Respir Crit Care Med 2015;192:958-64. [Crossref] [PubMed]
7. Kaukonen KM, Bailey M, Pilcher D, et al. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med 2015;372:1629-38. [Crossref] [PubMed]
8. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:801-10. [Crossref] [PubMed]
9. Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med 2003;31:1250-6. [Crossref] [PubMed]
10. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 2016;315:762-74. [Crossref] [PubMed]
11. Vincent JL, Martin GS, Levy MM. qSOFA does not replace SIRS in the definition of sepsis. Crit Care 2016;20:210. [Crossref] [PubMed]
12. Morgan RJ, Williams F, Wright MM. An early warning scoring system for detecting developing critical illness. Clin Intensive Care 1997;8:100.
13. National Institute of Health and Clinical Excellence. Acutely ill adults in hospital: recognising and responding to deterioration. 2007:1-30. Available online: https://www.nice.org.uk/guidance/CG50
14. National Confidential Enquiry into Patient Outcome and Death. Emergency admissions: a jouney in the right direction? 2007. Available online: http://www.ncepod.org.uk/2007report1/Downloads/EA_report.pdf
15. Jansen JO, Cuthbertson BH. Detecting critical illness outside the ICU: the role of track and trigger systems. Curr Opin Crit Care 2010;16:184-90. [Crossref] [PubMed]
16. Royal College of Physicians. National Early Warning Score (NEWS): Standardising the assessment of acuteillness severity in the NHS. Report of a working party. London: RCP, 2012. Available online: https://www.rcplondon.ac.uk/file/32/download?token=vfwDKQVS
17. Subbe CP, Kruger M, Rutherford P, et al. Validation of a modified Early Warning Score in medical admissions. QJM 2001;94:521-6. [Crossref] [PubMed]
18. Chen LM, Render M, Sales A, et al. Intensive care unit admitting patterns in the Veterans Affairs health care system. Arch Intern Med 2012;172:1220-6. [Crossref] [PubMed]
19. Gerlach H, Dhainaut JF, Harbarth S, et al. The PIRO concept: R is for response. Crit Care 2003;7:256-9. [Crossref] [PubMed]
20. Churpek MM, Snyder A, Han X, et al. qSOFA, SIRS, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients Outside the ICU. Am J Respir Crit Care Med 2016. [Epub ahead of print]. [Crossref] [PubMed]
21. Malbrain ML, Marik PE, Witters I, et al. Fluid overload, de-resuscitation, and outcomes in critically ill or injured patients: a systematic review with suggestions for clinical practice. Anaesthesiol Intensive Ther 2014;46:361-80. [Crossref] [PubMed]