Despite effort of control, healthcare-associated infections (HAIs) are still major health threats worldwide (1-6). Surveillance and feedback of HAIs rates to clinicians and other stakeholders is a cornerstone of infection prevention programs and also is recognized as one of core components comprise manageable and widely applicable ways to prevent HAIs and improve patients’ safety (3). Traditional surveillance methods involving manual surveillance by infection control practitioners (ICPs) for data collection processes are laborious, inefficient, and generate data of variable quality. It has been reported that up to 36–45% of infection prevention staff time is dedicated to undertaking surveillance (7-9). Developments in information technology have propelled a movement toward the use of standardized electronic surveillance system (ESS) in assisting ICPs in improving the efficacy of HAIs detecting (10,11) and 23–56% of facilities in USA have ESS (12). ESS was clearly encouraged to adopt in items of tertiary hospital certification (THC) to assist ICP in HAIs surveillance. HAI automatic surveillance and interaction platform (SIPS) is one of the most popular ESS and widespread chosen (13). Despite widespread availability, there is still absence of understanding barriers to implementing SIPS. In this study, we sought to evaluate the impact of SIPS for HAIs surveillance compared to manual survey in tertiary general hospitals.
We conducted a multi-center study in China with 21 tertiary general hospitals (13 academic and 8 non-academic centers). All recruited hospitals have adopted SIPS to monitor HAIs. Detailed flow diagram of SIPS was demonstrated in Figure 1. The SIPS would collect suspected cases for ICPs, then ICPs and clinicians online interacted and confirmed HAIs finally (Figure 1). This study included three stages and was approved by Institutional Review Board (No. 2019-SR-083) and each hospital received permission to participate in this study and sign a cooperation agreement. HAIs complied with CDC/NHSN surveillance definition of HAIs and criteria for specific types of infections in the acute care setting. In the first stage, a cross-sectional study was performed to investigate all the characters of hospitals and SIPS (beds, settings, year of ESS installment, HAI warning strategy and ESS problems). In the second stage. We selected indicators [incidence rate of HAIs, rate of miss-report HAIs, incidence rate of MRSA, CRE, CRAB and CRPA, incidence rate of central line-associated bloodstream infections (CLABSI), incidence rate of ventilator associated pneumonia (VAP) and incidence rate of catheter associated urine tract infections (CA-UTI)], which released by National Health Commission in 2015 (14). A retrospective before-after study of SIPS strategy was conducted to compare these indicators changes after SIPS installed (indicators after ESS one whole year vs. indicators before ESS one whole year). In the last stage, we performed a prospective study in 63 wards from 21 hospitals [20 neurosurgical wards, 19 general intensive care units (ICUs), 15 hematology wards, 6 neurology wards, 2 surgical ICUs and 1 vascular surgery ward]. All consecutive cases were judged manually by senior physician/ICP and the patient’s attending physician to determine whether they belonged to HAIs (gold standard) while the SIPS were applied to monitor the same cases in parallel. The incidence rate was calculated as the number of patients with HAIs divided by total number of beds and expressed per 100 beds with 95% confidence interval (CI). We calculated sensitivity and specificity for SIPS testing of HAIs. Summary receiver operating characteristic (SROC) curves were used to summarize the diagnostic accuracy of the results (15), and the area under the curve (AUC) was estimated to evaluate the diagnostic performance. AUC values of ≥0.97, 0.93–0.96, and 0.75–0.92 were considered to be excellent, very good and good diagnostic accuracy, respectively. I2 statistic was used as the preferred measure of variance to describe the heterogeneity of total variation in study and the random effects model approach was selected as study heterogeneity because of the variance. Potential sources of heterogeneity were investigated by meta-regression. Stata/SE 15.1 for Windows (College Station, TX, USA) and Review Manager software (Version 5.3, The Nordic Cochrane Centre) were used for data analysis.
The flowchart of SIPS was shown in Figure 1. In all, 21 tertiary general hospitals and 63 wards participated in this study. In the retrospective study, there were 27,030 HAIs among 1,143,457 patients in SIPS group and 16,791 HAIs among 938,117 patients in Without-SIPS group, and SIPS would significantly assist to detect more 1.5-fold new HAIs cases [odd ratio (OR) =1.50, 95% CI, 1.14–1.96]. In the subgroup study, we found SIPS would detect more 1.64-fold incidence of MRSA (OR =1.64, 95% CI, 1.05–2.56), 1.98-fold CRE (OR =1.98, 95% CI, 1.12–3.53), 2.21-fold CRAB (OR =2.21, 95% CI, 1.46–3.37) and 1.39-fold CR-UTI infection (OR =1.39, 95% CI, 1.01–1.90) and decrease 58% miss report rate of HAIs (OR =0.42, 95% CI, 0.30–0.59) (Table 1). In the prospective study, we collected 4,098 consecutive patients in 21 hospitals with 63 wards. The pooled sensitivity and specificity of SIPS for HAIs were 0.73 (95% CI, 0.67–0.78) and 0.81 (95% CI, 0.75–0.86), respectively (Figure 2) while there was significantly heterogeneity. The SROC curve revealed an AUC of 0.81 (95% CI, 0.77–0.84) (Figure 3). To reveal the sources of heterogeneity in this study, we performed a meta-regression analysis with the covariates including “early warning strategy difference (imaging examination, body temperature, serum inflammatory bio-markers, etc.)” “study areas” “beds of hospitals” “install year of SIPS” “wards (ICU or non-ICU)”, and “sample size” were assessed. We found that all of them showed significant influence on heterogeneity.
The inter-quartile range (IQR) of time saving identified varied from 50% to 90% (median: 76%), while all selected hospitals have some comments for SIPS, such as slow maintenance and frequent vulnerabilities (52.38%, 11/21 hospitals), unstable maintenance staff (71.43%, 15/21 hospitals), service attitude problem (33.33%, 7/21 hospitals) and high maintenance costs (42.86%, 9/21 hospitals).
Ten years ago, nosocomial infection monitoring was mainly manual and inefficient (16). For example, ICPs retrieved the microbial report from the laboratory, and then judged whether the patient has a nosocomial infection according to the results of microbial isolation and identification. However, there was a large underestimation of the risk of HAIs. ESS were wildly utilized to understand the nosocomial infections development (17-22). To date, our study firstly adopted large sample, multi-center studies to overall assess the impact of SIPS in the diagnosis of HAIs. Our study demonstrated that SIPS significantly improve ICPs work efficiency, detecting more HAIs which was consistent with Du et al.’s findings which SIPS assisted ICPs to deal with 70 new suspicious HAIs cases in one large volume hospital with 3,500 inpatients each day (13). Moreover, in the subgroup study, we found SIPS would significantly detect more 1.64-fold MRSA, 1.98-fould CRE, 2.21-fould CRAB and 1.39-fold CR-UTI and decrease 58% miss report rate of HAIs.
In the prospective study, we found SIPS maintained high levels of sensitivity (0.73, 95% CI, 0.67–0.78) and specificity (0.81, 95% CI, 0.75–0.86), and yields considerable dividends in ICPs staff time (median 76%, 95% CI, 50–90%). Our data demonstrated that adopting SIPS considerably improved the capacity for HAIs surveillance for ICPs staff. Interestingly, in the meta-regression study, we found that there was significant heterogeneity in sensitivity and specificity which affected by regions, hospital scale (bed number), system installation time, early warning strategy and wards. Meanwhile, SIPS was a purely commercial software that requires a lot of manpower, material and financial resources to update and maintain timely (23,24). In this case, SIPS has shortcomings in collecting data as a regional HAI monitoring platform, resulting in insufficient inter-regional comparability which should be considered carefully.
This study is the first large-scale multi-center study in tertiary general hospitals in China to comprehensively evaluate the effectiveness of SIPS. We demonstrated that SIPS significantly improved ICPs efficiency and HAIs monitoring effectiveness, but there were shortcomings such as untimely maintenance and high cost. In the choice of monitoring software of HAIs, the hospital needs to fully consider the scale, volume, monitoring purposes, the characteristics of the target population and the defect of the software itself (23,25).
Thank all support for this study: First People’s Hospital of Changzhou, People’s Hospital of Gaoyou, Second People’s Hospital of Huai’an, Affiliated Hospital of Jiangsu University, Northern Jiangsu Province Hospital, Kaifeng Second People’s Hospital, First People’s Hospital of Lianyungang, Luoyang Central Hospital, Affiliated Hospital of Nantong University, First People’s Hospital of Nantong, Qidong People’s Hospital, First Affiliated Hospital of Xiamen, Second Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Taixing People’s Hospital, Taizhou People’s Hospital, Wuxi No.2 People’s Hospital, Xi’an First Hospital, Yancheng First People’s Hospital, Affiliated hospital of Yangzhou University, People’s Hospital of Changshou District in Chongqing.
Funding: The present study was supported by grants from the National key Research & Development plan of Ministry of Science and Technology of the People’s Republic of China (grant no. 2018YFC1314900, 2018YFC1314901), the 2016 Industry Prospecting and Common Key Technology Key Projects of Jiangsu Province Science and Technology Department (grant no. BE2016002-4), the 2017 Projects of Jiangsu Provincial Department of Finance (grant no. 2150510), the 2016 Projects of Nanjing Science Bureau (Grant no. 201608003).
Conflicts of Interest: The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was approved by Institutional Review Board (No. 2019-SR-083) and each hospital received permission to participate in this study and sign a cooperation agreement. Written informed consent was obtained from all patients.
- Allegranzi B, Bagheri Nejad S, Combescure C, et al. Burden of endemic health-care-associated infection in developing countries: systematic review and meta-analysis. Lancet 2011;377:228-41. [Crossref] [PubMed]
- Suetens C, Latour K, Karki T, et al. Prevalence of healthcare-associated infections, estimated incidence and composite antimicrobial resistance index in acute care hospitals and long-term care facilities: results from two European point prevalence surveys, 2016 to 2017. Euro Surveill 2018. [Crossref] [PubMed]
- Zingg W, Holmes A, Dettenkofer M, et al. Hospital organisation, management, and structure for prevention of health-care-associated infection: a systematic review and expert consensus. Lancet Infect Dis 2015;15:212-24. [Crossref] [PubMed]
- Magill SS, Edwards JR, Bamberg W, et al. Multistate point-prevalence survey of health care-associated infections. N Engl J Med 2014;370:1198-208. [Crossref] [PubMed]
- Zahraei SM, Eshrati B, Masoumi Asl H, et al. Epidemiology of four main nosocomial infections in Iran during March 2007 - March 2008 based on the findings of a routine surveillance system. Arch Iran Med 2012;15:764-6. [PubMed]
- Lahsaeizadeh S, Jafari H, Askarian M. Healthcare-associated infection in Shiraz, Iran 2004-2005. J Hosp Infect 2008;69:283-7. [Crossref] [PubMed]
- Stone PW, Dick A, Pogorzelska M, et al. Staffing and structure of infection prevention and control programs. Am J Infect Control 2009;37:351-7. [Crossref] [PubMed]
- Mitchell BG, Hall L, Halton K, et al. Time spent by infection control professionals undertaking healthcare associated infection surveillance: A multi-centred cross sectional study. Infection Disease & Health 2016;21:36-40. [Crossref]
- Li Y, Gong Z, Lu Y, et al. Impact of nosocomial infections surveillance on nosocomial infection rates: A systematic review. Int J Surg 2017;42:164-9. [Crossref] [PubMed]
- van Mourik MSM, Perencevich EN, Gastmeier P, et al. Designing Surveillance of Healthcare-Associated Infections in the Era of Automation and Reporting Mandates. Clin Infect Dis 2018;66:970-6. [Crossref] [PubMed]
- Tseng YJ, Wu JH, Lin HC, et al. Rule-based healthcare-associated bloodstream infection classification and surveillance system. Stud Health Technol Inform 2013;186:145-9. [PubMed]
- Freeman R, Moore LS, Garcia Alvarez L, et al. Advances in electronic surveillance for healthcare-associated infections in the 21st Century: a systematic review. J Hosp Infect 2013;84:106-19. [Crossref] [PubMed]
- Du M, Xing Y, Suo J, et al. Real-time automatic hospital-wide surveillance of nosocomial infections and outbreaks in a large Chinese tertiary hospital. BMC Med Inform Decis Mak 2014;14:9. [Crossref] [PubMed]
- National Health and Health Commission. Available online: http://www.moh.gov.cn/yzygj/s7657/201504/5fa7461c3d044cb6a93eb6cc6eece087.shtml
- Rosman AS, Korsten MA. Application of summary receiver operating characteristics (sROC) analysis to diagnostic clinical testing. Adv Med Sci 2007;52:76-82. [PubMed]
- Quan TP, Hope R, Clarke T, et al. Using linked electronic health records to report healthcare-associated infections. PLoS One 2018;13:e0206860. [Crossref] [PubMed]
- Kramer TS, Schroder C, Behnke M, et al. Decrease of methicillin resistance in Staphylococcus aureus in nosocomial infections in Germany-a prospective analysis over 10 years. J Infect 2019;78:215-9. [Crossref] [PubMed]
- Hur EY, Jin Y, Jin T, et al. Development and Evaluation of the Automated Risk Assessment System for Catheter-Associated Urinary Tract Infection. Comput Inform Nurs 2019. [Epub ahead of print]. [Crossref] [PubMed]
- Yan Z, Zhou Y, Du M, et al. Prospective investigation of carbapenem-resistant Klebsiella pneumonia transmission among the staff, environment and patients in five major intensive care units, Beijing. J Hosp Infect 2019;101:150-7. [Crossref] [PubMed]
- Estañ-Capell J, Alarcón-Torres B, Bermúdez JD, et al. Effect of a surveillance system for decreasing neonatal nosocomial infections. Early Hum Dev 2019;131:36-40. [Crossref] [PubMed]
- Fasugba O, Koerner J, Bennett N, et al. Development and evaluation of a website for surveillance of healthcare-associated urinary tract infections in Australia. J Hosp Infect 2018;99:98-102. [Crossref] [PubMed]
- Hebert C, Flaherty J, Smyer J, et al. Development and validation of an automated ventilator-associated event electronic surveillance system: A report of a successful implementation. Am J Infect Control 2018;46:316-21. [Crossref] [PubMed]
- Gastmeier P, Behnke M. Electronic surveillance and using administrative data to identify healthcare associated infections. Curr Opin Infect Dis 2016;29:394-9. [Crossref] [PubMed]
- de Bruin JS, Seeling W, Schuh C. Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review. J Am Med Inform Assoc 2014;21:942-51. [Crossref] [PubMed]
- Tvardik N, Kergourlay I, Bittar A, et al. Accuracy of using natural language processing methods for identifying healthcare-associated infections. Int J Med Inform 2018;117:96-102. [Crossref] [PubMed]