Evaluation of manual and electronic healthcare-associated infections surveillance: a multi-center study with 21 tertiary general hospitals in China
Original Article

Evaluation of manual and electronic healthcare-associated infections surveillance: a multi-center study with 21 tertiary general hospitals in China

Wen-Sen Chen1#, Wei-Hong Zhang2#, Zhan-Jie Li1, Yue Yang1, Fu Chen3, Xue-Shun Ge4, Ting-Rui Wang5, Ping Fang6, Cheng-Yi Feng7, Jing Liu8, Shan-Shan Liu8, Hong-Xia Pan9, Tie-Lin Zhu10, Yuan-Yuan Tian11, Wen-Yi Wang12, Hu Xing13, Jing Yao13, Yong-Mei Yuan14, Ping Jiang15, Hong-Ping Tang16, Jun Zhou16, Jin-Cheng Zang17, Shan Lu18, Hui-Ping Huang19, Xiao-Hang Lei20, Bing-Hua Huang21, Shi-Hao Wang21, Feng-Yi Huang22, Hong-Ying Tao22, Yong-Xiang Zhang1, Bo Liu1, Hui-Fen Li1, Song-Qin Li1, Bi-Jie Hu23, Yun Liu24,25

1Department of Infection Control, First Affiliated Hospital of Nanjing Medical University. Nanjing 210029, China;2Department of Infection Control, Shengze Branch of Jiangsu Province Hospital & Jiangsu Shengze Hospital, Suzhou 215000, China;3Department of Infection Control, Northern Jiangsu Province Hospital, Yangzhou 225001, China;4Department of Infection Control, People’s Hospital of Gaoyou, Yangzhou 225600, China;5Department of Infection Control, Affiliated Hospital of Yangzhou University, Yangzhou 225000, China;6Department of Infection Control, Second People’s Hospital of Huai’an, Huai’an 223002, China;7Department of Infection Control, First People’s Hospital of Changzhou, Changzhou 213003, China;8Department of Infection Control, First People’s Hospital of Lianyungang, Lianyungang 222000, China;9Department of Infection Control, Taixing People’s Hospital, Taizhou 225400, China;10Department of Infection Control, Taizhou People’s Hospital, Taizhou 225400, China;11Department of Infection Control, Wuxi No.2 People’s Hospital, Wuxi 214000, China;12Department of Infection Control, Yancheng First People’s Hospital, Yancheng 224005, China;13Department of Infection Control, Affiliated Hospital of Jiangsu University, Zhenjiang 212001, China;14Department of Infection Control, Affiliated Hospital of Nantong University, Nantong 226001, China;15Department of Infection Control, First People’s Hospital of Nantong, Nantong 226001, China;16Department of Infection Control, People’s hospital of Qidong, Nantong 226200, China;17Department of Infection Control, Luoyang Central Hospital, Luoyang 471009, China;18Department of Infection Control, Kaifeng Second People’s Hospital, Kaifeng 475000, China;19Department of Infection Control, First Affiliated Hospital of Xiamen, Xiamen 361003, China;20Department of Infection Control, Xi’an First Hospital, Xi’an 710002, China;21Department of Infection Control, Second Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan 250000, China;22Department of Infection Control, People’s Hospital of Changshou District in Chongqing, Chongqing 401220, China;23Department of Infectious Disease and and Infection Control, Zhongshan Hospital, Fudan University, Shanghai 200000, China;24Department of Geriatrics Endocrinology, First Affiliated Hospital of Nanjing Medical University, Nanjing 210029, China;25School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing 210096, China

Contributions: (I) Conception and design: WS Chen, BJ Hu; (II) Administrative support: WH Zhang, YX Zhang, Y Liu; (III) Provision of study materials or patients: F Chen, XS Ge, TR Wang, P Fang, CY Feng, J Liu, SS Liu, HX Pan, TL Zhu, YY Tian, WY Wang, H Xing, J Yao, YM Yuan, P Jiang, HP Tang, J Zhou, JC Zang, S Lu, HP Huang, XH Lei, BH Huang, SH Wang, FY Huang, HY Tao; (IV) Collection and assembly of data: ZJ Li, Y Yang; (V) Data analysis and interpretation: B Liu, HF Li, SQ Li; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

#These authors contributed equally to this work.

Correspondence to: Yun Liu, MD, PhD. Department of Geriatrics Endocrinology, First Affiliated Hospital of Nanjing Medical University, Nanjing 210029, China. Email: liuyun@njmu.edu.cn.

Background: Healthcare-associated infections (HAIs) are still a major health threats worldwide. Traditional surveillance methods involving manual surveillance by infection control practitioners (ICPs) for data collection processes are laborious, inefficient, and generate data of variable quality. In this study, we sought to evaluate the impact of surveillance and interaction platform system (SIPS) for HAIs surveillance compared to manual survey in tertiary general hospitals.

Methods: A large multi-center study including 21 tertiary general hospitals and 63 wards were performed to evaluate the impact of electronic SIPS for HAIs.

Results: We collected 4,098 consecutive patients and found that the hospitals installed with SIPS significantly increased work efficiency of ICPs achieving satisfactory diagnostic performance of HAIs with 0.73 for sensitivity, 0.81 for specificity and 0.81 area under the curve (AUC). However, there were significant heterogeneity own to regions, time of SIPS installation, departments and sample size.

Conclusions: SIPS significantly improved ICPs efficiency and HAIs monitoring effectiveness, but there were shortcomings such as untimely maintenance and high cost.

Keywords: Healthcare-associated infections (HAIs); surveillance and interaction platform system (SIPS)

Submitted Aug 07, 2019. Accepted for publication Aug 15, 2019.

doi: 10.21037/atm.2019.08.80

Introduction

Despite effort of control, healthcare-associated infections (HAIs) are still major health threats worldwide (1-6). Surveillance and feedback of HAIs rates to clinicians and other stakeholders is a cornerstone of infection prevention programs and also is recognized as one of core components comprise manageable and widely applicable ways to prevent HAIs and improve patients’ safety (3). Traditional surveillance methods involving manual surveillance by infection control practitioners (ICPs) for data collection processes are laborious, inefficient, and generate data of variable quality. It has been reported that up to 36–45% of infection prevention staff time is dedicated to undertaking surveillance (7-9). Developments in information technology have propelled a movement toward the use of standardized electronic surveillance system (ESS) in assisting ICPs in improving the efficacy of HAIs detecting (10,11) and 23–56% of facilities in USA have ESS (12). ESS was clearly encouraged to adopt in items of tertiary hospital certification (THC) to assist ICP in HAIs surveillance. HAI automatic surveillance and interaction platform (SIPS) is one of the most popular ESS and widespread chosen (13). Despite widespread availability, there is still absence of understanding barriers to implementing SIPS. In this study, we sought to evaluate the impact of SIPS for HAIs surveillance compared to manual survey in tertiary general hospitals.

Methods

We conducted a multi-center study in China with 21 tertiary general hospitals (13 academic and 8 non-academic centers). All recruited hospitals have adopted SIPS to monitor HAIs. Detailed flow diagram of SIPS was demonstrated in Figure 1. The SIPS would collect suspected cases for ICPs, then ICPs and clinicians online interacted and confirmed HAIs finally (Figure 1). This study included three stages and was approved by Institutional Review Board (No. 2019-SR-083) and each hospital received permission to participate in this study and sign a cooperation agreement. HAIs complied with CDC/NHSN surveillance definition of HAIs and criteria for specific types of infections in the acute care setting. In the first stage, a cross-sectional study was performed to investigate all the characters of hospitals and SIPS (beds, settings, year of ESS installment, HAI warning strategy and ESS problems). In the second stage. We selected indicators [incidence rate of HAIs, rate of miss-report HAIs, incidence rate of MRSA, CRE, CRAB and CRPA, incidence rate of central line-associated bloodstream infections (CLABSI), incidence rate of ventilator associated pneumonia (VAP) and incidence rate of catheter associated urine tract infections (CA-UTI)], which released by National Health Commission in 2015 (14). A retrospective before-after study of SIPS strategy was conducted to compare these indicators changes after SIPS installed (indicators after ESS one whole year vs. indicators before ESS one whole year). In the last stage, we performed a prospective study in 63 wards from 21 hospitals [20 neurosurgical wards, 19 general intensive care units (ICUs), 15 hematology wards, 6 neurology wards, 2 surgical ICUs and 1 vascular surgery ward]. All consecutive cases were judged manually by senior physician/ICP and the patient’s attending physician to determine whether they belonged to HAIs (gold standard) while the SIPS were applied to monitor the same cases in parallel. The incidence rate was calculated as the number of patients with HAIs divided by total number of beds and expressed per 100 beds with 95% confidence interval (CI). We calculated sensitivity and specificity for SIPS testing of HAIs. Summary receiver operating characteristic (SROC) curves were used to summarize the diagnostic accuracy of the results (15), and the area under the curve (AUC) was estimated to evaluate the diagnostic performance. AUC values of ≥0.97, 0.93–0.96, and 0.75–0.92 were considered to be excellent, very good and good diagnostic accuracy, respectively. I2 statistic was used as the preferred measure of variance to describe the heterogeneity of total variation in study and the random effects model approach was selected as study heterogeneity because of the variance. Potential sources of heterogeneity were investigated by meta-regression. Stata/SE 15.1 for Windows (College Station, TX, USA) and Review Manager software (Version 5.3, The Nordic Cochrane Centre) were used for data analysis.

Figure 1 Flow diagram demonstrating the SIPS. SIPS, surveillance and interaction platform system; HAI, healthcare-associated infection; HASIP, healthcare-associated infection automatic surveillance and interaction platform; EHRs, Electronic health records; AOS, anesthesia operation system; HIS, hospital information system; ICP, infection control practitioner; LIS, laboratory information system; PACS, picture achieving and communication system.

Results

The flowchart of SIPS was shown in Figure 1. In all, 21 tertiary general hospitals and 63 wards participated in this study. In the retrospective study, there were 27,030 HAIs among 1,143,457 patients in SIPS group and 16,791 HAIs among 938,117 patients in Without-SIPS group, and SIPS would significantly assist to detect more 1.5-fold new HAIs cases [odd ratio (OR) =1.50, 95% CI, 1.14–1.96]. In the subgroup study, we found SIPS would detect more 1.64-fold incidence of MRSA (OR =1.64, 95% CI, 1.05–2.56), 1.98-fold CRE (OR =1.98, 95% CI, 1.12–3.53), 2.21-fold CRAB (OR =2.21, 95% CI, 1.46–3.37) and 1.39-fold CR-UTI infection (OR =1.39, 95% CI, 1.01–1.90) and decrease 58% miss report rate of HAIs (OR =0.42, 95% CI, 0.30–0.59) (Table 1). In the prospective study, we collected 4,098 consecutive patients in 21 hospitals with 63 wards. The pooled sensitivity and specificity of SIPS for HAIs were 0.73 (95% CI, 0.67–0.78) and 0.81 (95% CI, 0.75–0.86), respectively (Figure 2) while there was significantly heterogeneity. The SROC curve revealed an AUC of 0.81 (95% CI, 0.77–0.84) (Figure 3). To reveal the sources of heterogeneity in this study, we performed a meta-regression analysis with the covariates including “early warning strategy difference (imaging examination, body temperature, serum inflammatory bio-markers, etc.)” “study areas” “beds of hospitals” “install year of SIPS” “wards (ICU or non-ICU)”, and “sample size” were assessed. We found that all of them showed significant influence on heterogeneity.

Table 1
Table 1 SIPS significantly detected more HAI retrospective studies in 21 hospitals
Full table
Figure 2 SIPS vs. manual survey for HAIs in a multi-center study with 21 tertiary general hospitals and 63 wards. SIPS, surveillance and interaction platform system; HAI, healthcare-associated infection.
Figure 3 SROC curve of SIPS vs. manual survey for HAIs in a multi-center study with 21 tertiary general hospitals and 63 wards. SIPS, surveillance and interaction platform system; HAI, healthcare-associated infection; SROC, summary receiver operating characteristic.

The inter-quartile range (IQR) of time saving identified varied from 50% to 90% (median: 76%), while all selected hospitals have some comments for SIPS, such as slow maintenance and frequent vulnerabilities (52.38%, 11/21 hospitals), unstable maintenance staff (71.43%, 15/21 hospitals), service attitude problem (33.33%, 7/21 hospitals) and high maintenance costs (42.86%, 9/21 hospitals).

Discussion

Ten years ago, nosocomial infection monitoring was mainly manual and inefficient (16). For example, ICPs retrieved the microbial report from the laboratory, and then judged whether the patient has a nosocomial infection according to the results of microbial isolation and identification. However, there was a large underestimation of the risk of HAIs. ESS were wildly utilized to understand the nosocomial infections development (17-22). To date, our study firstly adopted large sample, multi-center studies to overall assess the impact of SIPS in the diagnosis of HAIs. Our study demonstrated that SIPS significantly improve ICPs work efficiency, detecting more HAIs which was consistent with Du et al.’s findings which SIPS assisted ICPs to deal with 70 new suspicious HAIs cases in one large volume hospital with 3,500 inpatients each day (13). Moreover, in the subgroup study, we found SIPS would significantly detect more 1.64-fold MRSA, 1.98-fould CRE, 2.21-fould CRAB and 1.39-fold CR-UTI and decrease 58% miss report rate of HAIs.

In the prospective study, we found SIPS maintained high levels of sensitivity (0.73, 95% CI, 0.67–0.78) and specificity (0.81, 95% CI, 0.75–0.86), and yields considerable dividends in ICPs staff time (median 76%, 95% CI, 50–90%). Our data demonstrated that adopting SIPS considerably improved the capacity for HAIs surveillance for ICPs staff. Interestingly, in the meta-regression study, we found that there was significant heterogeneity in sensitivity and specificity which affected by regions, hospital scale (bed number), system installation time, early warning strategy and wards. Meanwhile, SIPS was a purely commercial software that requires a lot of manpower, material and financial resources to update and maintain timely (23,24). In this case, SIPS has shortcomings in collecting data as a regional HAI monitoring platform, resulting in insufficient inter-regional comparability which should be considered carefully.

Conclusions

This study is the first large-scale multi-center study in tertiary general hospitals in China to comprehensively evaluate the effectiveness of SIPS. We demonstrated that SIPS significantly improved ICPs efficiency and HAIs monitoring effectiveness, but there were shortcomings such as untimely maintenance and high cost. In the choice of monitoring software of HAIs, the hospital needs to fully consider the scale, volume, monitoring purposes, the characteristics of the target population and the defect of the software itself (23,25).

Acknowledgments

Thank all support for this study: First People’s Hospital of Changzhou, People’s Hospital of Gaoyou, Second People’s Hospital of Huai’an, Affiliated Hospital of Jiangsu University, Northern Jiangsu Province Hospital, Kaifeng Second People’s Hospital, First People’s Hospital of Lianyungang, Luoyang Central Hospital, Affiliated Hospital of Nantong University, First People’s Hospital of Nantong, Qidong People’s Hospital, First Affiliated Hospital of Xiamen, Second Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Taixing People’s Hospital, Taizhou People’s Hospital, Wuxi No.2 People’s Hospital, Xi’an First Hospital, Yancheng First People’s Hospital, Affiliated hospital of Yangzhou University, People’s Hospital of Changshou District in Chongqing.

Funding: The present study was supported by grants from the National key Research & Development plan of Ministry of Science and Technology of the People’s Republic of China (grant no. 2018YFC1314900, 2018YFC1314901), the 2016 Industry Prospecting and Common Key Technology Key Projects of Jiangsu Province Science and Technology Department (grant no. BE2016002-4), the 2017 Projects of Jiangsu Provincial Department of Finance (grant no. 2150510), the 2016 Projects of Nanjing Science Bureau (Grant no. 201608003).

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was approved by Institutional Review Board (No. 2019-SR-083) and each hospital received permission to participate in this study and sign a cooperation agreement. Written informed consent was obtained from all patients.

References

  1. Allegranzi B, Bagheri Nejad S, Combescure C, et al. Burden of endemic health-care-associated infection in developing countries: systematic review and meta-analysis. Lancet 2011;377:228-41. [Crossref] [PubMed]
  2. Suetens C, Latour K, Karki T, et al. Prevalence of healthcare-associated infections, estimated incidence and composite antimicrobial resistance index in acute care hospitals and long-term care facilities: results from two European point prevalence surveys, 2016 to 2017. Euro Surveill 2018. [Crossref] [PubMed]
  3. Zingg W, Holmes A, Dettenkofer M, et al. Hospital organisation, management, and structure for prevention of health-care-associated infection: a systematic review and expert consensus. Lancet Infect Dis 2015;15:212-24. [Crossref] [PubMed]
  4. Magill SS, Edwards JR, Bamberg W, et al. Multistate point-prevalence survey of health care-associated infections. N Engl J Med 2014;370:1198-208. [Crossref] [PubMed]
  5. Zahraei SM, Eshrati B, Masoumi Asl H, et al. Epidemiology of four main nosocomial infections in Iran during March 2007 - March 2008 based on the findings of a routine surveillance system. Arch Iran Med 2012;15:764-6. [PubMed]
  6. Lahsaeizadeh S, Jafari H, Askarian M. Healthcare-associated infection in Shiraz, Iran 2004-2005. J Hosp Infect 2008;69:283-7. [Crossref] [PubMed]
  7. Stone PW, Dick A, Pogorzelska M, et al. Staffing and structure of infection prevention and control programs. Am J Infect Control 2009;37:351-7. [Crossref] [PubMed]
  8. Mitchell BG, Hall L, Halton K, et al. Time spent by infection control professionals undertaking healthcare associated infection surveillance: A multi-centred cross sectional study. Infection Disease & Health 2016;21:36-40. [Crossref]
  9. Li Y, Gong Z, Lu Y, et al. Impact of nosocomial infections surveillance on nosocomial infection rates: A systematic review. Int J Surg 2017;42:164-9. [Crossref] [PubMed]
  10. van Mourik MSM, Perencevich EN, Gastmeier P, et al. Designing Surveillance of Healthcare-Associated Infections in the Era of Automation and Reporting Mandates. Clin Infect Dis 2018;66:970-6. [Crossref] [PubMed]
  11. Tseng YJ, Wu JH, Lin HC, et al. Rule-based healthcare-associated bloodstream infection classification and surveillance system. Stud Health Technol Inform 2013;186:145-9. [PubMed]
  12. Freeman R, Moore LS, Garcia Alvarez L, et al. Advances in electronic surveillance for healthcare-associated infections in the 21st Century: a systematic review. J Hosp Infect 2013;84:106-19. [Crossref] [PubMed]
  13. Du M, Xing Y, Suo J, et al. Real-time automatic hospital-wide surveillance of nosocomial infections and outbreaks in a large Chinese tertiary hospital. BMC Med Inform Decis Mak 2014;14:9. [Crossref] [PubMed]
  14. National Health and Health Commission. Available online: http://www.moh.gov.cn/yzygj/s7657/201504/5fa7461c3d044cb6a93eb6cc6eece087.shtml
  15. Rosman AS, Korsten MA. Application of summary receiver operating characteristics (sROC) analysis to diagnostic clinical testing. Adv Med Sci 2007;52:76-82. [PubMed]
  16. Quan TP, Hope R, Clarke T, et al. Using linked electronic health records to report healthcare-associated infections. PLoS One 2018;13:e0206860. [Crossref] [PubMed]
  17. Kramer TS, Schroder C, Behnke M, et al. Decrease of methicillin resistance in Staphylococcus aureus in nosocomial infections in Germany-a prospective analysis over 10 years. J Infect 2019;78:215-9. [Crossref] [PubMed]
  18. Hur EY, Jin Y, Jin T, et al. Development and Evaluation of the Automated Risk Assessment System for Catheter-Associated Urinary Tract Infection. Comput Inform Nurs 2019. [Epub ahead of print]. [Crossref] [PubMed]
  19. Yan Z, Zhou Y, Du M, et al. Prospective investigation of carbapenem-resistant Klebsiella pneumonia transmission among the staff, environment and patients in five major intensive care units, Beijing. J Hosp Infect 2019;101:150-7. [Crossref] [PubMed]
  20. Estañ-Capell J, Alarcón-Torres B, Bermúdez JD, et al. Effect of a surveillance system for decreasing neonatal nosocomial infections. Early Hum Dev 2019;131:36-40. [Crossref] [PubMed]
  21. Fasugba O, Koerner J, Bennett N, et al. Development and evaluation of a website for surveillance of healthcare-associated urinary tract infections in Australia. J Hosp Infect 2018;99:98-102. [Crossref] [PubMed]
  22. Hebert C, Flaherty J, Smyer J, et al. Development and validation of an automated ventilator-associated event electronic surveillance system: A report of a successful implementation. Am J Infect Control 2018;46:316-21. [Crossref] [PubMed]
  23. Gastmeier P, Behnke M. Electronic surveillance and using administrative data to identify healthcare associated infections. Curr Opin Infect Dis 2016;29:394-9. [Crossref] [PubMed]
  24. de Bruin JS, Seeling W, Schuh C. Data use and effectiveness in electronic surveillance of healthcare associated infections in the 21st century: a systematic review. J Am Med Inform Assoc 2014;21:942-51. [Crossref] [PubMed]
  25. Tvardik N, Kergourlay I, Bittar A, et al. Accuracy of using natural language processing methods for identifying healthcare-associated infections. Int J Med Inform 2018;117:96-102. [Crossref] [PubMed]
Cite this article as: Chen WS, Zhang WH, Li ZJ, Yang Y, Chen F, Ge XS, Wang TR, Fang P, Feng CY, Liu J, Liu SS, Pan HX, Zhu TL, Tian YY, Wang WY, Xing H, Yao J, Yuan YM, Jiang P, Tang HP, Zhou J, Zang JC, Lu S, Huang HP, Lei XH, Huang BH, Wang SH, Huang FY, Tao HY, Zhang YX, Liu B, Li HF, Li SQ, Hu BJ, Liu Y. Evaluation of manual and electronic healthcare-associated infections surveillance: a multi-center study with 21 tertiary general hospitals in China. Ann Transl Med 2019;7(18):444. doi: 10.21037/atm.2019.08.80

Article Options

Download Citation

Share