Predicting subretinal fluid absorption with machine learning in patients with central serous chorioretinopathy

Introduction

Central serous chorioretinopathy (CSC) is a retinochoroid disease that causes idiopathic serous retinal detachment, which is associated with one or more leakages from the choroid through the defects in the retinal pigment epithelium (RPE) outer blood-retina barrier. It primarily affects relatively young men of working age (1,2). CSC is fairly common, being considered as the fourth most prevalent non-surgical retinopathy associated with subretinal fluid (SRF) leakage in the world (3). Although SRF can resolve spontaneously in some cases, many patients still suffer permanent vision loss or significant clinical sequelae due to incomplete subretinal fluid absorption (SFA) (2).

CSC is usually divided into two categories: acute CSC and chronic CSC, based on the duration of symptoms. Most investigators have employed this incomplete and relatively rudimentary classification of CSC in their clinical studies (2). However, there is no clear consensus regarding the criteria for classification. Our lack of an established classification system necessitates studying the natural disease progression of CSC and its therapeutic management (4,5). There remains an absence of academically recognized treatment guidelines for CSC. Ophthalmologists have to make decisions experientially in the treatment of CSC patients.

To develop more precise care for patients, we have established an individualized management plan based on SFA utilizing big data. SFA is the most concerning issue for clinicians after treatment, and it is the most important prognostic characteristic for patients with CSC (6). The increase or decrease in SFA affects the therapeutic strategy and the follow-up intervals for patients. In our study, we tried to establish an intelligent prediction system to foresee SFA at 1, 3 and 6 months after laser treatment with big data that incorporates medical records and imaging features, which help us to clarify the prognosis of patients with CSC and choose a sensible treatment.

We present the following article in accordance with the TRIPOD reporting checklist (available at http://dx.doi.org/10.21037/atm-20-1519).

Methods

Data collection

CSC was defined as patients with leakage on their fundus fluorescein angiography (FFA), abnormal choroidal circulation such as hyperpermeability, dilated choroidal vessels or other abnormal microangiopathy on indocyanine green angiography (ICGA), and serous retinal detachment (SRD) as confirmed by optical coherence tomography angiography (OCTA) and optical coherence tomography (OCT). A total of 416 eyes in 401 patients and 64 eyes in 60 patients were studied at Zhongshan Ophthalmic Center (ZOC) and Xiamen Eye Center (XEC), respectively, from January 2013 to September 2019. A definite diagnosis was made for all patients based on FFA and ICGA. The patients were followed for 1–6 months after treatment. Study exclusion criteria were as follows: (I) patients with high myopia, as defined as a refractive error (spherical equivalent) 26.5 mm and (II) patients with media opacities or signal strength indexes who were affected. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the ethics board of ZOC (No. 2020KYPJ024) and individual consent for this retrospective analysis was waived.

A total of 6,732 imaging pictures (1,248 FFA, 1,248 ICGA, 1,412 OCTA, and 2,824 OCT images) and 554 imaging pictures (192 FFA and 362 OCT images) were collected from ZOC and XEC, respectively. FFA (Heidelberg Spectralis, Heidelberg, Germany) and ICGA (Heidelberg Spectralis, Heidelberg, Germany) images for each patient were included only at the baseline, including three images from the early, middle and late phases. However, for the OCTA (RTVue XR Avanti with AngioVue; Optovue Inc., Fremont, CA, USA) and OCT (Heidelberg Spectralis, Heidelberg, Germany) follow-up data, data from the baseline, 1, 3, and 6 months after laser treatment were included. Measurement information on the FFA, ICGA, OCTA, and OCT, were extracted from the software of Heidelberg Eye Explorer (version 1.7.1.0) and Optovue (Version 2017.1.0.155). The clinical features (20 clinical features, e.g., the duration of CSC) of these CSC patients were also extracted from electronic medical records (details are provided in Table S1). For therapy information, the data from ZOC included conventional laser (CL) therapy (117 eyes), subthreshold micropulse laser (SML) therapy (80 eyes) and half-dose photodynamic therapy (hd-PDT) (219 eyes). The XEC data included CL therapy (21 eyes), 577nm SML therapy (14 eyes) and hd-PDT (29 eyes). According to type 3 of the TRIPOD statement, we developed prediction models using the dataset from ZOC and evaluated its performance in a separate dataset from XEC (7).

We treated each eye as a separate CSC case during data preprocessing. There were only a few values missing from the ZOC data, and we filled them in with the mean values for other cases. However, all the ICGA and OCTA features were missing for patients collected from XEC data. Considering their clinical significance and importance in the algorithm, we ultimately chose to fill in the missing features in the XEC data with the mean values for the same features of the ZOC data.

Construction of models

All the training and testing approaches were run on a workstation configured with 32-core Intel Xeon E5 CPU and 128 GB RAM. We used Python 3.6.8 in the Ubuntu 16.04 system. The Python libraries we used in this study are as follows: Jupyter (1.1.0), Scikit-Learn (0.19.1), and Pandas (0.20.3). Six separate algorithms were trained, with a total of 165 features and state-of-the-art performance in each adaptive domain, and they are listed as follows: Decision Tree (8), AdaBoost.R2 (9), Gradient Boosting (10), XGBoost (11), random forest (12), and extra-trees (13). While selecting the optimal algorithms, we randomly divided the ZOC data into 10 parts, and calculated the decision tree, AdaBoost.R2, gradient boosting, XGBoost, random forest, and extra-trees by 10-fold cross-validation. We then selected the best three algorithms for the ensemble according to the prediction accuracy. After determining the selected algorithms, to make the most of the existing clinical data, we used all the data from ZOC to retrain the selected algorithms. The workflow is shown in Figure 1.

Figure 1 Overall Study Workflow. Workflow diagram showing the training overview for the SFA prediction model.

For each of the above algorithms, we used a grid search with cross-validation to select the suitable hyperparameters (14).

The ensemble learning method was applied to obtain a model with good fitting ability and generalization performance for tasks such as classification and regression and model averaging is a common and effective approach (14,15).

Evaluation of models

To evaluate the performance of our models, the accuracy in predicting SFA at 1, 3 and 6 months after laser treatment were validated. The baseline data were used to predict SFA at 1, 3 and 6 months after treatment. To obtain more accurate predictions, when predicting SFA at 3 months after treatment, we trained the model using the baseline and 1-month data; to predict SFA at 6 months after treatment, we trained the models using the baseline, 1-month and 3-month data. A 10-fold cross-validation was applied to evaluate the performance of the models.

Simplified model

We constructed a simplified prediction model using relatively few clinical data and OCT features to make our study more accessible for clinical use. The remaining features were determined according to the relative importance obtained during the establishment of the original algorithms (Figures S1-S6), and the difficulty in imaging feature acquisition. Table S2 shows all the training features of the simplified model. For the simplified model, the training steps are the same as those in the original models.

Statistical analysis

Accuracy (ACC) is to evaluate the predictive effectiveness of the model. Values are shown as means ± SDs.

Results

A total of 480 eyes in 461 patients aged 28 to 71 years old (43.56±6.64) were addressed during our study. The demographic information for the training and validation datasets are shown in Table 1. Table 2 shows the accuracies of predicting SFA during all the algorithm tasks. Among the original models, random forest had the best performance in the internal validation, and XGBoost performed best at external validation. For the simplified models, gradient boosting had the best performance for internal validation, and the blending algorithm performed best at external validation.

Table 1 Patient demographics
Full table

Table 2 Accuracy of the subretinal fluid absorption predictions during internal and external validation tests
Full table

During the internal validation, random forest performed best in predicting SFA, with accuracies of 0.651±0.068, 0.753±0.065 and 0.818±0.058 at 1, 3 and 6 months, respectively. In the external validation, XGBoost performed best in predicting SFA, with accuracies of 0.734, 0.727, and 0.900 at 1, 3 and 6 months, respectively. The simplified model showed a comparable level of predictive power. In the internal validation, gradient boosting performed best at predicting SFA, with accuracies of 0.630±0.057, 0.780±0.043 and 0.818±0.074 at 1, 3 and 6 months, respectively. During the external validation, the blending model performed best at predicting SFA, with accuracies of 0.656, 0.758, and 0.900 at 1, 3 and 6 months, respectively.

In the cross-validation, random forest achieved high-accuracy predictions with areas under the curve (AUCs) ranging from 0.35 to 0.80 for 1 month, from 0.63 to 0.89 for 3 months, and from 0.92 to 1.00 for 6 months. In the external validation, XGBoost provided high-accuracy predictions with AUCs ranging from 0.76 to 1.00 for 1 month, from 0.27 to 0.72 for 3 months, and from 0.98 to 1.00 for 6 months (Figure 2). The simplified model exhibited an analogous prediction accuracy with AUCs ranging from 0.57 to 0.81 for 1 month, from 0.23 to 0.97 for 3 months, and from 0.88 to 1.00 for 6 months in the cross-validation, and AUCs ranging from 0.72 to 1.00 for 1 month, from 0.27 to 0.66 for 3 months, and from 0.98 to 1.00 for 6 months in the external validation (Figure 3). The distributions of prediction results and ground truth in each task are revealed in the confusion matrixes (CM) of Figure 2 and Figure 3. As shown in the receiver operating characteristic curve (ROC curve) and the CM, the SFA prediction at 6 months after treatment was the most accurate. Figures S1-S6 show the importance of features in the SFA predictions at 1, 3 and 6 months.

Figure 2 Prediction performance in the internal and external validation tests on the full model. Panels A, B, and C, CM of the classification in the internal validation test. Panels D, E, and F, ROC of the internal validation test. Panels G, H, and I, CM of the classification in the external validation test. Panels J, K, and L, ROC of the external validation test. CM, confusion matrix; ROC, receiver operating characteristic curve.

Figure 3 Prediction performance in the internal and external validation tests on the simplified model. Panels A, B, and C, CM of the classification in the internal validation test. Panels D, E, and F, ROC of the internal validation test. Panels G, H, and I, CM of the classification in the external validation test. Panels J, K, and L, ROC of the external validation test. CM, confusion matrix; ROC, receiver operating characteristic curve.

Discussion

To our knowledge, no study has previously generated a machine learning model for predicting SFA in CSC patients. Our prediction models can foresee the patient’s condition six months in advance. However, recent predictions within three months are relatively imprecise. Generally, short-term targets should be more accurate than long-term targets for applying artificial intelligence in disease prognosis prediction (16). This may be due to the lower limitation of the courses in the inclusion criteria; as the baseline data for CSC patients vary greatly, and the short-term consistency of patient prognosis is low. Usually, at six months after treatment, the SFA rates of CSC patients were much higher than at one and three months, which makes prediction tasks easier (17,18). Last but not least, for prognosis predictions of complex fundus disease, we still need to accumulate follow-up data to improve our models.

By predicting SFA in patients, we can better understand the progression of CSC, choose cost-effective therapies and manage the follow-up more efficiently. The increasing use of FFA, ICGA, OCTA, and OCT in studying CSC has greatly improved our understanding of its pathogenesis and imaging characteristics. However, there is still no clear consensus regarding the criteria for classification and the guidelines for treatment. Our predictive system provides a reference for clinicians to choose therapies. We can input the information related to the patients and different therapies, and then the models will give us the probability of presenting SFA in six months. This is a new strategy for mining big data, and it enables us to achieve precise treatment without considering the specific classification of the disease. Based on the prediction models established in this study, we could choose a more efficient method such as hd-PDT in patients with consistent SRF, and we can also choose a more economical therapy such as SML for patients with SRF that is easily absorbed.

More than helping clinicians choose reasonable therapies, the models also help define the factors that are relevant to the CSC prognosis. In the analysis of feature importance, we found that in addition to the therapies chosen by clinicians, the baseline characteristics on the retina and lifestyle also exert a significant impact on SFA, including the SRF height, the central macula thickness (CMT), the double-layer sign (DLS), and the scores on the Pittsburgh Sleep Quality Index and Hamilton Anxiety Scale (19,20). The findings can help us analyze the relevant factors that lead to different prognoses in CSC patients with the same treatment and guide patients to pay attention to influencing these factors in their daily life.

To make our study applicable to different scenarios, we constructed a simplified prediction model according to the relative importance obtained in the original models and the accessibility of the imaging features; the model is trained with only 11 clinical features and OCT features. This advantage can be further appreciated in applications by hospitals in underdeveloped areas without FFA, ICGA, and OCTA. In a clinical setting, FFA, ICGA, and OCTA are not necessary at certain stages, in most cases.

Limitations

There are some limitations in our present study. To improve the accuracy of the short-term predictions within three months, we need to incorporate the duration of the disease into the inclusion criteria. More data on CSC patients are necessary to improve the accuracy of the SFA prediction models. In addition, data for external validation is unicentric, and more real-world tests are needed for improved accuracy.

Conclusions

In summary, our study showed that multidimensional patterns of clinical and imaging features are predictive factors of SFA in CSC patients. The prediction models provide us with a whole new strategy to counsel patients from an individual-based perspective, and they serve as references for ophthalmologists who can choose efficient therapies and make follow-up schedules.

Acknowledgments

Funding: We received funding from the National Key R&D Program of China (2018YFC0116500), the National Natural Science Foundation of China (81670866), the Major Project of Guangzhou Science and Technology Committee (201707020008), the National Natural Science Foundation of China (81770967), the National Natural Science Foundation of China (81822010), and the Jiangsu Planned Projects for Postdoctoral Research Funds (2019K226). The sponsors and funding organizations

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at http://dx.doi.org/10.21037/atm-20-1519

Data Sharing Statement: Available at http://dx.doi.org/10.21037/atm-20-1519

Peer Review File: Available at http://dx.doi.org/10.21037/atm-20-1519

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm-20-1519). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the study in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the ethics board of ZOC (No. 2020KYPJ024) and individual consent for this retrospective analysis was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Wong KH, Lau KP, Chhablani J, et al. Central serous chorioretinopathy: what we have learnt so far. Acta Ophthalmol 2016;94:321-5. [Crossref] [PubMed]
2. van Rijssen TJ, van Dijk EHC, Yzer S, et al. Central serous chorioretinopathy: Towards an evidence-based treatment guideline. Prog Retin Eye Res 2019;73:100770. [Crossref] [PubMed]
3. Wang M, Munch IC, Hasler PW, et al. Central serous chorioretinopathy. Acta Ophthalmol 2008;86:126-45. [Crossref] [PubMed]
4. Mohabati D, van Rijssen TJ, van Dijk EH, et al. Clinical characteristics and long-term visual outcome of severe phenotypes of chronic central serous chorioretinopathy. Clin Ophthalmol 2018;12:1061-70. [Crossref] [PubMed]
5. Otsuka S, Ohba N, Nakao K. A long-term follow-up study of severe variant of central serous chorioretinopathy. Retina 2002;22:25-32. [Crossref] [PubMed]
6. Manayath GJ, Ranjan R, Karandikar SS, et al. Central serous chorioretinopathy: Current update on management. Oman J Ophthalmol 2018;11:200-6. [Crossref] [PubMed]
7. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2015;350:g7594. [Crossref] [PubMed]
8. Schoning V, Hammann F. How far have decision tree models come for data mining in drug discovery? Expert Opin Drug Discov 2018;13:1067-9. [Crossref] [PubMed]
9. Qi Z, Meng F, Tian Y, et al. Adaboost-LLP: A Boosting Method for Learning With Label Proportions. IEEE Trans Neural Netw Learn Syst 2018;29:3548-59. [Crossref] [PubMed]
10. Luo Y, Ye W, Zhao X, et al. Classification of Data from Electronic Nose Using Gradient Tree Boosting Algorithm. Sensors (Basel) 2017;17:2376. [Crossref] [PubMed]
11. Ogunleye AA, Qing-Guo W. XGBoost Model for Chronic Kidney Disease Diagnosis. IEEE/ACM Trans Comput Biol Bioinform 2019. [Epub ahead of print].
12. Pavey TG, Gilson ND, Gomersall SR, et al. Field evaluation of a random forest activity classifier for wrist-worn accelerometer data. J Sci Med Sport 2017;20:75-80. [Crossref] [PubMed]
13. Nattee C, Khamsemanan N, Lawtrakul L, et al. A novel prediction approach for antimalarial activities of Trimethoprim, Pyrimethamine, and Cycloguanil analogues using extremely randomized trees. J Mol Graph Model 2017;71:13-27. [Crossref] [PubMed]
14. Baskin II. Machine Learning Methods in Computational Toxicology. Methods Mol Biol 2018;1800:119-39. [Crossref] [PubMed]
15. Zhang Y, Li M, Han S, et al. Intelligent Identification for Rock-Mineral Microscopic Images Using Ensemble Machine Learning Algorithms. Sensors (Basel) 2019;19:3914. [Crossref] [PubMed]
16. Rohm M, Tresp V, Muller M, et al. Predicting Visual Acuity by Using Machine Learning in Patients Treated for Neovascular Age-Related Macular Degeneration. Ophthalmology 2018;125:1028-36. [Crossref] [PubMed]
17. Zhou L, Chong V, Lai K, et al. A pilot prospective study of 577-nm yellow subthreshold micropulse laser treatment with two different power settings for acute central serous chorioretinopathy. Lasers Med Sci 2019;34:1345-51. [Crossref] [PubMed]
18. van Dijk EHC, Fauser S, Breukink MB, et al. Half-Dose Photodynamic Therapy versus High-Density Subthreshold Micropulse Laser Treatment in Patients with Chronic Central Serous Chorioretinopathy: The PLACE Trial. Ophthalmology 2018;125:1547-55. [Crossref] [PubMed]
19. Manzar MD. Dimensionality of the Pittsburgh Sleep Quality Index: a systematic review. Health Qual Life Outcomes 2018;16:89. [Crossref] [PubMed]
20. Maier W, Buller R, Philipp M, et al. The Hamilton Anxiety Scale: reliability, validity and sensitivity to change in anxiety and depressive disorders. J Affect Disord 1988;14:61-8. [Crossref] [PubMed]