Evaluation of the updated “Candida score” with Sepsis 3.0 criteria in critically ill patients

Introduction

Invasive Candida infections (ICIs) are the most common invasive fungal infections, constituting 70–90% of all invasive mycoses (1); they are associated with high mortality, especially in intensive care units (ICUs) (2-4). Unfortunately, early diagnosis of ICI remains a challenge. Numerous risk factors for ICI have been identified (5). Some clinical prediction rules were developed and validated to identify ICU patients at high risk of ICI (6,7). In 2006, a Spanish group, using the database of the Estudio de Prevalencia de CANdidiasis project, identified and validated four predictors of proven ICIs and proposed the “Candida score” to identify patients with high risk of ICI; this score was calculated as 1 × (total parenteral nutrition) + 1 × (surgery) + 1 × (multifocal Candida colonization) + 2 × (severe sepsis) (8,9).

However, the Candida score proposed in 2006 used the concept of severe sepsis, which was based on systemic inflammatory response syndrome (SIRS) diagnosis; this concept of severe sepsis was removed in the new Sepsis 3.0 definition and diagnostic criteria in 2016 (10). Furthermore, organ dysfunction, which was described with the dynamic change of the Sequential Organ Failure Assessment (SOFA) score (11), has been identified as an important risk factor for ICI (12,13). Therefore, the Candida score should be updated with the Sepsis 3.0 definition and undergo appropriate validation.

In this study, we investigated the clinical relevance of the Candida score with the Sepsis 3.0 definition (CS-3.0), compared with the Candida score with the classic severe sepsis definition (CS-2009) in newly admitted critically ill patients. We hypothesized that organ dysfunction, evaluated with the SOFA score, would be clinically useful for ICI prediction and that the updated CS-3.0 would exhibit better prediction performance. We present the following article in accordance with the STROBE reporting checklist (available at http://dx.doi.org/10.21037/atm-20-995).

Methods

Study design

This was a retrospective analysis of a large cohort of critically ill patients. According to the classic sepsis diagnostic criteria, patients with infection who met two or more SIRS criteria and showed evidence of organ dysfunction, hypoperfusion, or hypotension during the first day after ICU admission were regarded as severe sepsis cases. And according to the Sepsis 3.0 definition, patients with infection and SOFA score ≥2 were regarded as Sepsis 3.0 cases (10,14).

Data source

This retrospective analysis used data collected from the MIMIC-III open source clinical database (version 1.4, released on September 2, 2016), which was developed and maintained by the Massachusetts Institute of Technology, Philips Healthcare, and Beth Israel Deaconess Medical Center (15). Information derived from the electronic medical records of 46,476 unique critical care patients, admitted to the ICUs at Beth Israel Deaconess Medical Center between 2001 and 2012, was included in this freely accessible database (16). MIMIC-III data are compliant with the Health Insurance Portability and Accountability Act of 1996. Use of the MIMIC-III database was approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center and Massachusetts Institute of Technology, and a waiver of informed consent was granted.

Patients

All patients in the database were screened, while for patients with multiple ICU stays, only data related to the first ICU admission were considered. The criteria for inclusion in this study were as follows: (I) adults (≥18 years of age) at the time of ICU admission; (II) ICU stay ≥72 hours. Exclusion criteria were neutropenia (i.e., a total leukocyte count <500/mm³ at the first complete blood count test after ICU admission) and pregnancy.

Candida score

Components of CS-2009 included severe sepsis, total parenteral nutrition, surgery, and multifocal Candida colonization, while CS-3.0 replaced the concept of “severe sepsis” with the Sepsis 3.0 definition while its weight remains as 2 points (10). Other components were defined according to criteria proposed by León et al. (8). The Candida score was calculated by adding points provided by each component, as proposed by León et al. (8,9); this included two points for severe sepsis (CS-2009) or Sepsis 3.0 (CS-3.0), and one point for each remaining variable including total parenteral nutrition, initial surgery, and multifocal Candida colonization. Initial surgery patients were identified based on whether the patients underwent surgery before ICU admission; other components were calculated based on medical records data during the first 72 hours after ICU admission. Candida colonization was considered multifocal when Candida species were concurrent isolated from two or more noncontiguous foci, even if the species differed among the foci.

Outcome

Primary outcome was defined as diagnosis with ICI using standard criteria, in accordance with the method reported by León et al. (8,9). Briefly, candidemia was diagnosed based on at least one blood culture positive for Candida, while candida peritonitis was diagnosed based on isolation or positive culture for Candida from peritoneal fluid collected during surgical procedures. Isolation of Candida species from normally sterile body fluids, such as pleural fluid or abscess fluid, was also regarded as diagnostic of invasive candidiasis. Other clinical outcomes were recorded, including hospital mortality and ICU and hospital stays, as well as durations of vasopressor and ventilation use.

Statistical analysis

Data are expressed as the mean ± standard deviation, median with interquartile range, and proportions (absolute and relative frequencies), as appropriate. Student’s t-test or the Mann-Whitney U test was used to compare continuous variables; the χ² test or Fisher’s exact test was used to compare categorical variables. The discriminatory powers of CS-2009 and CS-3.0 were evaluated by the area under the receiver operating characteristic curve; the improvement in area under the receiver operating characteristic curve after updating Candida score was evaluated by Delong’s test (17). Sensitivities, specificities, predictive positive and negative values, and relative risks were estimated, along with their respective 95% confidence intervals. Except that the Delong’s test was performed in R software (version 3.3.0, https://www.r-project.org) and pROC package (http://expasy.org/tools/pROC/), all other statistical analyses were performed by using IBM^® SPSS^® Statistics, version 24 (IBM Corp., Armonk, NY, USA). Differences with P<0.05 were considered statistically significant.

Results

Baseline characteristics of the study cohort

This retrospective study of patients in the MIMIC-III database included 17,666 patients (from a total of 58,976 hospital admissions); 436 patients (2.5%) were diagnosed with ICI, as shown in Figure 1. Compared with non-ICI patients, the patients with ICI had higher proportions of medical ICU admission and emergency admission, as well as higher Simplified Acute Physiology Score II and SOFA scores; patients with ICI also had higher median CS-2009 scores [2 (range, 1–3) vs. 3 (range, 3–4)], higher median CS-3.0 scores [2 (range, 1–3) vs. 3 (range, 3–4)], and higher proportions of positive Candida score risk factors (total parenteral nutrition, initial surgery, and multifocal Candida colonization), as shown in Table 1. The proportions of patients who met severe sepsis or Sepsis 3.0 criteria both significantly differ between ICI and non-ICI groups (severe sepsis: 65.7% vs. 78.4%, P<0.005; Sepsis 3.0: 66.0% vs. 81.2%, P<0.005), while the proportions of patients who met two or more SIRS criteria did not (91.6% vs. 92.5%, P=0.551). Besides, in the ICI group, the number of patients who met the Sepsis 3.0 criteria was greater than the number of patients with severe sepsis (81.2% vs. 78.4%, P<0.005).

Figure 1 Flowchart of the patients included in this study.

Table 1 Characteristics of patients with evidence of sepsis at intensive care unit admission
Full table

Analysis of the microbiological characteristics of patients with ICI showed that, with increasing CS-3.0 score, diversity increased in terms of sources of infection and categories of Candida (Table 2). In patients with CS-3.0 scores ≤4, more than half of ICIs were caused by Candidemia (CS-3.0 <3 points: 53.5%; 3 points: 62.1%; 4 points: 51%). In patients with CS-3.0 scores of 4–5, ICIs were more often associated with abdominal infection (CS-3.0 4 points: 25%, 5 points: 51.8%). In all of the subgroups, half of the ICIs were due to Candida albicans (CS-3.0 <3 points: 51.5%; 3 points: 50.8%; 4 points: 51.9%; 5 points: 51.8%). Among the non-Candida albicans isolations, the proportion of Candida glabrata contributed the largest one and it rose from 17.2% to 26.8% as the CS-3.0 increased from <3 to 5.

Table 2 Microbiological characteristics of patients with invasive candidiasis according to the Candida score
Full table

Discrimination performance and predictive accuracy of CS-2009 and CS-3.0 models

In the study cohort, the areas under the receiver operating characteristic curves of the CS-2009 and CS-3.0 models were 0.789 (95% confidence interval, 0.765–0.813) and (95% confidence interval, 0.782–0.827), respectively. The performance characteristics of the two models are summarized in Table 3; the CS-2009 results were similar to those demonstrated in the original study (9). The DeLong test showed that differences in areas under the receiver operating characteristic curves between CS-2009 and CS-3.0 models were statistically significant (Delong’s P=0.03) (Figure 2).

Table 3 Discriminatory powers of Candida score 2009 and Candida score 3.0 in the validation cohort
Full table

Figure 2 Receiver operating characteristic curves of CS-2009 and CS-3.0 models.

Subgroup analyses of CS-2009 and CS-3.0 models

Subgroup analysis showed that in all subgroups with CS-3.0 scores ≥3, there were smaller proportions of patients with severe sepsis, compared with the proportions of patients who met the Sepsis 3.0 criteria (Table 4). Moreover, SIRS diagnosis was predominant (>90%) in most subgroups and did not significantly differ among them; other Candida score parameters (e.g., meeting the Sepsis 3.0 criteria, whether patients underwent surgery before ICU admission (initial surgery), multifocal colonization, and total parenteral nutrition) showed remarkable differences among subgroups. Analysis of the patient distribution showed that the ICI rate increased concomitantly with the number of Candida score risk factors; furthermore, the CS-3.0 model predicted higher infection rates, compared with the rates predicted by the CS-2009 model, in all subgroups (Figure 3).

Table 4 Risk factors and severity scores of invasive candidiasis according to the Candida score
Full table

Figure 3 Distribution of overall patients and patients with invasive candidiasis according to the Candida score. Dark bars denote the CS-3.0 model, while gray bars denote the CS-2009 model. X-axis indicates stratification based on CS-2009 or CS-3.0 scores.

We also investigated the association between antifungal therapy initiated during the first 48 hours after ICU admission and the ICU outcomes of patients in this study. The results showed that the use of antifungal therapy was not associated with CS-3.0 score. Our results did not demonstrate significant benefits in terms of hospital mortality or length of hospital stay, due to the use of antifungal therapy during the first 48 hours after ICU admission; however, the length of ICU stay significantly differed between patients with ICI who had a CS-3.0 score of <3 and those who had a CS-3.0 score of 3 (Table 5). The most common agents used for antifungal therapy were azoles (e.g., voriconazole, fluconazole, posaconazole, and itraconazole) and echinocandins (e.g., caspofungin and micafungin). Amphotericin B (and its liposome) was also used for treatment of a few patients.

Table 5 Antifungal therapy during the first 48 hours after ICU admission and intensive care unit outcomes for patients with invasive candidiasis according to the Candida score
Full table

Discussion

In this study, we investigated the prevalence of ICI after ICU admission, and performed validation of both the classic CS-2009 model and the new CS-3.0 model. The overall prevalence of ICI in the study cohort was 2.5%, whereas it was 6.9% (CS-2009 ≥3 points) and 7.1% (CS-3.0 ≥3 points) in patients with a high risk of ICI. Considering the relatively low morbidity and prolonged culture/diagnosis period for ICI, a large cohort and thorough clinical record were necessary for this study. Using the large amount of data in MIMIC III v1.4, one of the largest public databases available regarding intensive care, our results confirmed that the CS-3.0 score is more accurate than the CS-2009 score for predicting the clinical diagnosis of ICI after ICU admission. To the best of our knowledge, this is the largest clinical study on Candida score validation; it is also the first to demonstrate that organ dysfunction and Sepsis 3.0 diagnosis are of considerable importance for predicting ICI in the critical care setting.

Based on relatively recent studies, intensivists have progressively recognized the significance of early identification of patients at high risk of ICI and the association between delayed antifungal therapy and increased mortality in these patients (18-20). Several research groups have proposed risk prediction models based upon clinical factors or Candida colonization parameters; among them, the Candida score has been validated as a useful tool to identify patients with high risk of ICI and to distinguish ICU patients who would benefit from early antifungal treatment (7-9,21). However, the original Candida score was based on the severe sepsis concept, which was non-quantitative; moreover, the diagnosis of SIRS, which was required for the diagnosis of classic sepsis, is non-specific in clinical settings. Therefore, the Sepsis-3 definition and diagnostic criteria (10) excluded SIRS and severe sepsis. The newly defined sepsis definition, based on dynamic changes in the SOFA score, should be used in a revised Candida score.

Previous researchers have expressed concern regarding the usefulness of sepsis definitions based on SIRS criteria, because the SIRS criteria are applicable to most critically ill patients admitted to the ICU (22). Traditionally, SIRS was regarded as an assessment criterion for systemic inflammation and was a fundamental requirement of the classic sepsis/severe sepsis concept. Nevertheless, in the critical care setting, various intervention methods and diversity in medical conditions among admitted patients (e.g., heart rate, mechanical ventilation, and temperature management) altered the inflammatory presentation of the potential infection; therefore, the SIRS criteria lacked adequate specificity and predictive validity (11). In the present study, we did not find significant differences in the incidence of SIRS between ICI and non-ICI groups. In subgroups with various Candida scores, the proportions of patients who met two or more SIRS criteria were greater than 90%, which confirmed that SIRS and the derived classic sepsis/severe sepsis concept were inadequate and insensitive for prediction of ICI.

In our study, the updated Candida score model, using the Sepsis 3.0 definition, showed superior prediction performance for ICI, compared with the classic model. SOFA scores were higher in patients with ICI and in patients with Candida score ≥3, which indicated that organ dysfunction is important for prediction of ICI; it also highlighted the critical role of the evaluation of organ dysfunction, both qualitative and quantitative, in the prediction of ICI (23). As shown in a previous study, organ dysfunction was critical in the treatment of ICI because of its association with clinical outcome and because of its impact on the use of antifungal agents; notably, the use of these agents may be independently associated with mortality in patients with ICI (24).

In addition to the prediction of ICI, the usefulness of some risk-factor-based predictive methods has been suggested for empirical antifungal treatment and has shown validity and cost-effectiveness (25,26). However, our retrospective analysis did not show a significant association between the use of antifungal therapy during the first 48 hours after ICU admission and clinical outcomes in patients with ICI, except with respect to the length of ICU stay in the groups with CS <3 points and CS =3 points. It could be partially interpreted that most of ICI cases chose azoles as the antifungal therapy in the critical care settings, while in the Clinical Practice Guideline for the Management of Candidiasis updated in 2016 (27), the echinocandin is recommended as initial therapy for candidemia and suspected candidiasis in nonneutropenic patients. Notably, as the Candida score increased from 3 to 5, the proportion of echinocandin use also increased; this was presumably because of the concurrent increase in isolation of Candida glabrata, which may be a useful observation for clinical practice and further investigation.

Our study had several limitations. First, it was limited by its retrospective nature and the source of data used. Therefore, no causal relationships could be established between Candida score and ICI. Additionally, one of the Candida score risk factors, Candida colonization, could only be assessed in patients who underwent the corresponding evaluation after ICU admission. Second, our study only focused on the status at ICU admission, due to the use of a public database; it did not exclude patients who received empirical antifungal therapy before ICU admission. Because of the relatively long treatment course and the nature of these real-world data, we did not exclude those patients from the cohort and mainly focused on the treatment course after ICU admission. Thus, our results on antifungal therapy were different from that of empirical therapy in the previous studies and need further validated and investigated in the critical care settings.

Conclusions

Our study confirmed the clinical relevance and comparative superiority of the updated Candida score model, using the Sepsis 3.0 definition, compared with the classic sepsis/severe sepsis model, in assessment of critically ill patients who were newly admitted to the ICU. Considering the clinical importance of organ dysfunction in ICI, the Sepsis 3.0 should be used as the basis for prediction of ICI.

Acknowledgments

Funding: This study was supported by the Non-profit Central Research Institute Fund of Chinese Academy of Medical Sciences (No. 2019XK320040), the National Natural Science Foundation of China (No. 81601657) and Wu Jieping Medical Foundation (No. 320.6750.18419).

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at http://dx.doi.org/10.21037/atm-20-995

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm-20-995). NC received support for the present research from the Non-profit Central Research Institute Fund of Chinese Academy of Medical Sciences and the National Natural Science Foundation of China. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the ethical standards of the Declaration of Helsinki (as revised in 2013). Use of the MIMIC-III database was approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center and Massachusetts Institute of Technology, and a waiver of informed consent was granted. All authors gave approval of the final manuscript to be published.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Eggimann P, Garbino J, Pittet D. Epidemiology of Candida species infections in critically ill non-immunosuppressed patient. Lancet Infect Dis 2003;3:685-702. [Crossref] [PubMed]
2. Leleu G, Aegerter P, Guidet B. Systemic candidiasis in intensive care units: A multicenter, matched-cohort study. Journal of Critical Care 2002;17:168-75. [Crossref] [PubMed]
3. Leroy O, Gangneux JP, Montravers P, et al. Epidemiology, management, and risk factors for death of invasive Candida infections in critical care: A multicenter, prospective, observational study in France (2005–2006). Crit Care Med 2009;37:1612-8. [Crossref] [PubMed]
4. Montravers P, Mira JP, Gangneux JP, et al. A multicentre study of antifungal strategies and outcome of Candida spp. peritonitis in intensive-care units. Clin Microbiol Infect 2011;17:1061-7. [Crossref] [PubMed]
5. Guery BP, Arendrup MC, Auzinger G, et al. Management of invasive candidiasis and candidemia in adult non-neutropenic intensive care unit patients: Part I. Epidemiology and diagnosis. Intensive Care Med 2009;35:55-62. [Crossref] [PubMed]
6. Paphitou NI, Ostroskyzeichner L, Rex JH. Rules for identifying patients at increased risk for candidal infections in the surgical intensive care unit: approach to developing practical criteria for systematic use in antifungal prophylaxis trials. Med Mycol 2005;43:235-43. [Crossref] [PubMed]
7. Ostrosky-Zeichner L, Sable C, Sobel J, et al. Multicenter retrospective development and validation of a clinical prediction rule for nosocomial invasive candidiasis in the intensive care setting. Eur J Clin Microbiol Infect Dis 2007;26:271-6. [Crossref] [PubMed]
8. León C, Ruiz-Santana S, Saavedra P, et al. A bedside scoring system ("Candida score") for early antifungal treatment in nonneutropenic critically ill patients with Candida colonization. Crit Care Med 2006;34:730-7. [Crossref] [PubMed]
9. León C, Ruizsantana S, Saavedra P, et al. Usefulness of the "Candida score" for discriminating between Candida colonization and invasive candidiasis in non-neutropenic critically ill patients: a prospective multicenter study. Crit Care Med 2009;37:1624-33. [Crossref] [PubMed]
10. Shankar-Hari M, Phillips GS, Levy ML, et al. Developing a New Definition and Assessing New Clinical Criteria for Septic Shock. JAMA 2016;315:775-87. [Crossref] [PubMed]
11. Vincent JL, De MA, Cantraine F, et al. Use of the SOFA score to assess the incidence of organ dysfunction/failure in intensive care units: results of a multicenter, prospective study. Working group on "sepsis-related problems" of the European Society of Intensive Care Medicine. Crit Care Med 1998;26:1793-800. [Crossref] [PubMed]
12. Keighley C, Chen SC, Marriott D, et al. Candidaemia and a risk predictive model for overall mortality: a prospective multicentre study. BMC Infect Dis 2019;19:445. [Crossref] [PubMed]
13. Kato H, Yoshimura Y, Suido Y, et al. Mortality and risk factor analysis for Candida blood stream infection: A multicenter study. J Infect Chemother 2019;25:341-5. [Crossref] [PubMed]
14. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992;101:1644-55. [Crossref] [PubMed]
15. Johnson AE, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. Sci Data 2016;3:160035. [Crossref] [PubMed]
16. Johnson A, Pollard T, Mark R. MIMIC-III Clinical Database. Available online: http://dx.doi.org/ [Crossref]
17. DeLong ER, Delong DM, Clarke-Pearson DL. Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics 1988;44:837-45. [Crossref] [PubMed]
18. Kumar A, Roberts D, Wood KE, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006;34:1589-96. [Crossref] [PubMed]
19. Garey KW, Rege M, Pai MP, et al. Time to initiation of fluconazole therapy impacts mortality in patients with candidemia: a multi-institutional study. Clin Infect Dis 2006;43:25-31. [Crossref] [PubMed]
20. Morrell M, Fraser VJ, Kollef MH. Delaying the empiric treatment of Candida bloodstream infection until positive blood culture results are obtained: a potential risk factor for hospital mortality. Antimicrob Agents Chemother 2005;49:3640-5. [Crossref] [PubMed]
21. Leroy G, Lambiotte F, Thévenin D, et al. Evaluation of "Candida score" in critically ill patients: a prospective, multicenter, observational, cohort study. Ann Intensive Care 2011;1:50. [Crossref] [PubMed]
22. Sprung CL, Sakr Y, Vincent JL, et al. An evaluation of systemic inflammatory response syndrome signs in the Sepsis Occurrence In Acutely Ill Patients (SOAP) study. Intensive Care Med 2006;32:421-7. [Crossref] [PubMed]
23. Forstner C, Lassnigg A, Tobudic S, et al. A prospective analysis of invasive candidiasis following cardiac surgery: severity markers are predictive. J Infect 2013;66:528-35. [Crossref] [PubMed]
24. Cui N, Wang H, Qiu H, et al. Impact of initial empirical antifungal agents on the outcome of critically ill patients with invasive candidiasis: analysis of the China-SCAN study. Int J Antimicrob Agents 2017;50:74-80. [Crossref] [PubMed]
25. Faiz S, Neale B, Rios E, et al. Risk-based fluconazole prophylaxis of Candida bloodstream infection in a medical intensive care unit. Eur J Clin Microbiol Infect Dis 2009;28:689-92. [Crossref] [PubMed]
26. Group TS. Preliminary Results of a Multicenter, International, Retrospective, Study to Validate a Clinical Prediction Rule (CPR) to Identify Critically-Ill Patients at Risk of Invasive Candidiasis (IC) for TReatment with Empirical Antifungal Therapy (TREAT Study). Infectious Diseases Society of America. 2008.
27. Pappas PG, Kauffman CA, Andes DR, et al. Executive Summary: Clinical Practice Guideline for the Management of Candidiasis: 2016 Update by the Infectious Diseases Society of America. Clin Infect Dis 2016;62:409-17. [Crossref] [PubMed]