Diagnostic accuracy of cancer ratio for malignant pleural effusion: a systematic review and meta-analysis

Introduction

Pleural effusion (PE) is common in patients who visiting the department of emergency, respiratory or thoracic diseases (1). Etiologies of PE are various, with common causes of tuberculous pleural effusion (TPE), parapneumonic effusion (PPE), malignant pleural effusion (MPE), hear failure (HF) and others (2). The diagnosis of MPE is usually made with PE cytology or thoracentesis with pleural biopsy. Cytology is an inexpensive diagnostic tool with high specificity; however, its sensitivity is only around 0.6Jeny (3), depends on the size and stage of primary tumor. Pleural biopsy is widely used for MPE diagnosis; however, it is an invasive tool with some complications such as pain, subcutaneous emphysema and bleeding (2). In addition, its accuracy is greatly affected by the experience of operator and observer.

Biomarkers in PE have been considered to be useful tools for MPE diagnosis (4). Compared with biopsy and cytology, biomarker has some advantages, including inexpensiveness, mini-invasiveness and objective results. In 2016, Verma et al. reported that serum lactate dehydrogenase (LDH) to pleural fluid adenosine deaminase (ADA) ratio (cancer ratio, CR) has high diagnostic accuracy for MPE (5). Subsequently, several studies have been performed to validate the findings of Verma et al., but the results were various. Therefore, we performed this systematic review and meta-analysis to ascertain the diagnostic accuracy of CR for MPE.

Methods

Databases and search strategy

This systematic review and meta-analysis was performed in accordance with the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) guideline (6). We searched the PubMed and EMBASE databases to identify eligible studies up to 7 June, 2019. The searched algorithm in PubMed was: (((((("Lactate Dehydrogenase" OR LDH)) AND (ADA OR Adenosine Deaminase))) OR ("cancer ratio"))) AND (pleural). We used a similar strategy in searching EMBASE database. In addition, references listed at the end of relevant reviews, eligible studies were manually searched to identify potential studies.

Study selection

The searched literatures were imported into Endnote, a reference management software, to remove duplicates. All studies investigating the diagnostic accuracy of CR for MPE were included into this systematic review and meta-analysis. The exclusion criteria were: (I) animal studies; (II) non-English published studies; (III) studies with insufficient data to construct a two by two table; (IV) conference abstracts.

Two reviewers independently screened titles and abstracts of the retrieved studies. Disagreements were resolved with consensus or full text reviewing.

Data extraction and quality assessment

Two reviewers independently extracted following data from the eligible studies: name of the first author, publication year, sample size, components of control, type of data collection (prospective or retrospective), reference standard for MPE diagnosis, sensitivity, specificity, area under ROC curve (AUC) and corresponding threshold. We used sensitivity, specificity and sample sizes of MPE and control to construct a two by two table for further meta-analysis.

We used the revised tool for the QUADAS-2 to assess the quality of eligible studies (7). Any disagreements in data extraction and quality assessment were resolved with consensus.

Statistical analysis

We used a bivariate model to pool sensitivities and specificities of the eligible studies (8). A sROC curve was constructed to determine the overall diagnostic accuracy of CRJeny (9). Heterogeneity across all eligible studies was estimated with I² (10). We performed this meta-analysis with Stata 12.0, as recommended by Wang et al. (11). P value less than 0.05 was statistically significant (12).

Results

Study selection process and summary of eligible studies

Figure 1 is a flowchart depicting the study selection process. Finally, five studies with 596 MPEs and 863 BPEs were included (5). Characteristics of the eligible studies were summarized in Table 1. Two studies were from Singapore (5,13) and the remaining 3 studies were from China (14), Egypt (15) and Poland (16). Sample sizes of the eligible studies ranged from 60 to 987. All of the eligible studies, except one (13), set TPE and PPE as control. Prospective data collection was seen in two studies (13,14). Biopsy was used as reference standard in all studies and cytology was also used in three studies (5,13,14).

Figure 1 Flowchart depicting the study selection process.

Table 1 Summary of eligible studies
Full table

Design quality and major findings of eligible studies

Table 2 lists major findings of eligible studies. Four of five eligible studies reported that the AUC of CR is around 0.80, while one study with small sample size (n=60) reported that the AUC of CR is 1.00. Cut-off in three studies were 20 (5,13,15), and the remaining two were 16.4 (16) and 10.6 (14). Sensitivities in all eligible studies was higher than 0.94, and specificity ranged from 0.60 to 1.00.

Table 2 Major findings of eligible studies
Full table

Table 3 lists the quality of eligible studies. Because all of the eligible studies were not reported in accordance with the Standards for Reporting of Diagnostic Accuracy (STARD) guideline (17), some domains of QUADAS-2 tool were labeled as unknown. The patients selection bias domain in three studies was labeled as high because of retrospective design (5,16) or only patients with lymphocytic predominant exudative pleural effusion were included (13). All except one of the eligible studies did not report whether CR was determined without information of final diagnosis; therefore, the index text domain of these studies was labeled as unknown. The reference standard domain of all eligible studies was labeled as unknown because the authors did not report whether the reference standard (biopsy and cytology) was performed in all participants (partial verification bias). Following and timing domain of one study was labeled as high because patients without final diagnosis were excluded from final analysis (5).

Table 3 Quality assessment of eligible studies
Full table

Meta-analysis

The pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR) and diagnostic odds ratio (DOR) of CR for MPE were 0.97 (95% CI: 0.92–0.99), 0.89 (95% CI: 0.69–0.97), 8.64 (95% CI: 2.78–26.90), 0.04 (95% CI: 0.02–0.10) and 228.75 (95% CI: 32.73–1598.59), respectively. Significantly heterogeneity was observed across all included studies (I²=89.22, 95% CI: 80.21–98.23).

Figure 2 is a sROC curve for CR. The area under sROC curve was 0.98 (95% CI: 0.97–0.99).

Figure 2 sROC curve for CR. sROC, summary receiver operating characteristic; CR, cancer ratio.

Discussion

To the best of our knowledge, this is the first systematic review and meta-analysis estimating the diagnostic accuracy of CR for MPE. The major findings of this study were: (I) to date, five studies with 596 MPEs and 863 controls have investigated the diagnostic accuracy of CR for MPE; (II) the pooled sensitivity and specificity of CR were 0.97 (95% CI: 0.92–0.99) and 0.89 (95% CI: 0.69–0.97), respectively; (III) the major design weaknesses of included studies were patients selection bias and potential partial verification bias. Taken together, the current evidence supports that CR has extremely high diagnostic accuracy for MPE.

According to the Light’s criteria (18), PE can be categorized into transudate and exudate. MPE is a common cause of exudate. The other two common causes of exudate are TPE and PPE. Previous studies indicated that ADA in PE has high diagnostic accuracy for TPE (19). In addition, PPE patients have significantly higher PE ADA level than MPE (20). Therefore, low PE ADA level suggests high probability of MPE. On the other hand, previous studies also reported that LDH level in serum, rather than PE, is significantly higher in MPE than that in TPE and PPE (21,22), indicating that high serum LDH is associated with high probability of MPE. Therefore, serum LDH: PE ADA ratio represents a potential index for MPE diagnosis.

Sensitivity and specificity are two basic metrics of a diagnostic tool (23). This meta-analysis found that the sensitivity and specificity of CR were 0.97 and 0.89, respectively, indicating that CR can confirm 97% of MPE rule out 89% of non-MPE. The major limitation of sensitivity and specificity is that they only reflect the diagnostic accuracy of a tool at a certain threshold. Therefore, they are not a global measure of test efficiency (24). By contrast, area under sROC curve is not threshold dependent and thus usually used as a global measure of test efficiency (12). Area under sROC curve ranges from 0.5 to 1.0, with a higher value indicating higher accuracy (9). We found that the area under sROC curve was 0.98, indicating that CR has extremely high diagnostic accuracy for MPE. PLR and NLR are another two clinical meaningful indicators of a diagnostic tool. Likelihood ratios above 10 and below 0.1 are strong indicators of ruling in or ruling out the target disease in study cohort (25). We found that the PLR and NLR of CR were 8.64 and 0.04, respectively, indicating that positive CR is insufficient to rule in MPE but negative CR is sufficient to rule out MPE.

Some tumor makers in PE have been reported to be useful diagnostic tools for MPE, such as neuron-specific enolase (NSE), carcinoembryonic antigen (CEA), cancer antigen (CA) 15-3, CA 19-9, CA 125, cytokeratin fragment 19 (CYFRA 21-1). Previous meta-analyses have showed that the diagnostic accuracy of these tumor markers is moderate, with an area under sROC curve less than 0.90 (4,26-30). We found that the area under sROC curve of CR was 0.98, suggesting that the diagnostic accuracy of CR is higher than traditional tumor markers.

We assessed the quality of eligible studies and found that the available studies had some design weaknesses. The major design weakness of available studies was patients selection bias. Some of the eligible studies did not report whether the participants were consecutively enrolled. Therefore, the representativeness of study cohort is questionable. Indeed, all of the included studies set TPE and PPE as controls and other etiologies of exudate PE, such as chylothorax, chemical pleurisy or connective tissue disease, were not included. Future studies with more types of control should be performed to rigorously evaluate the diagnostic accuracy of CR.

Take together, our systematic review and meta-analysis indicates that CR has extremely high diagnostic accuracy for MPE. Given the design weaknesses of available studies, further studies with rigorous design are needed to validate the findings of this meta-analysis.

Acknowledgments

Funding: This work was supported by a grant from the National Natural Science Foundation of China (Grant Number 81860501). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

References

1. Porcel JM, Azzopardi M, Koegelenberg CF, et al. The diagnosis of pleural effusions. Expert Rev Respir Med 2015;9:801-15. [Crossref] [PubMed]
2. Wang XJ, Yang Y, Wang Z, et al. Efficacy and safety of diagnostic thoracoscopy in undiagnosed pleural effusions. Respiration 2015;90:251-5. [Crossref] [PubMed]
3. Assawasaksakul T, Boonsarngsuk V, Incharoen P. A comparative study of conventional cytology and cell block method in the diagnosis of pleural effusion. J Thorac Dis 2017;9:3161-7. [Crossref] [PubMed]
4. Yang Y, Liu YL, Shi HZ. Diagnostic Accuracy of Combinations of Tumor Markers for Malignant Pleural Effusion: An Updated Meta-Analysis. Respiration 2017;94:62-9. [Crossref] [PubMed]
5. Verma A, Abisheganaden J, Light RW. Identifying Malignant Pleural Effusion by A Cancer Ratio (Serum LDH: Pleural Fluid ADA Ratio). Lung 2016;194:147-53. [Crossref] [PubMed]
6. McInnes MDF, Moher D, Thombs BD, et al. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies. JAMA 2018;319:388-96. [Crossref] [PubMed]
7. Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529-36. [Crossref] [PubMed]
8. Reitsma JB, Glas AS, Rutjes AW, et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58:982-90. [Crossref] [PubMed]
9. Walter SD. Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data. Stat Med 2002;21:1237-56. [Crossref] [PubMed]
10. Higgins JPT, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ 2003;327:557-60. [Crossref] [PubMed]
11. Wang J, Leeflang M. Recommended software/packages for meta-analysis of diagnostic accuracy. J Lab Precis Med 2019;4:22. [Crossref]
12. Liu C, Cui YL, Ding CM, et al. Diagnostic accuracy of interferon-gamma in pericardial effusions for tuberculous pericarditis: a meta-analysis. J Thorac Dis 2018;10:854-60. [Crossref] [PubMed]
13. Verma A, Dagaonkar RS, Marshall D, et al. Differentiating Malignant from Tubercular Pleural Effusion by Cancer Ratio Plus (Cancer Ratio: Pleural Lymphocyte Count). Can Respir J 2016;2016:7348239. [Crossref] [PubMed]
14. Zhang F, Hu L, Wang J, et al. Clinical value of jointly detection serum lactate dehydrogenase/pleural fluid adenosine deaminase and pleural fluid carcinoembryonic antigen in the identification of malignant pleural effusion. J Clin Lab Anal 2017;31:e22106. [Crossref] [PubMed]
15. Elmahalawy II, Abdelaal GA, Alhanafy MA, et al. Role of cancer ratio and carcinogenic embryonic antigen ratio in diagnosis of pleural effusion. Egypt J Chest Dis Tuberc 2017;66:671-4. [Crossref]
16. Korczyński P, Mierzejewski M, Krenke R, et al. Cancer ratio and other new parameters for differentiation between malignant and nonmalignant pleural effusions. Pol Arch Intern Med 2018;128:354-61. [PubMed]
17. Bossuyt PM, Reitsma JB, Bruns DE, et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clin Chem 2003;49:7-18. [Crossref] [PubMed]
18. Light RW, Macgregor MI, Luchsinger PC, et al. Pleural effusions: the diagnostic separation of transudates and exudates. Ann Intern Med 1972;77:507-13. [Crossref] [PubMed]
19. Aggarwal AN, Agarwal R, Sehgal IS, et al. Adenosine deaminase for diagnosis of tuberculous pleural effusion: A systematic review and meta-analysis. PLoS One 2019;14:e0213728. [Crossref] [PubMed]
20. Chung W, Jung Y, Lee K, et al. CXCR3 ligands in pleural fluid as markers for the diagnosis of tuberculous pleural effusion. Int J Tuberc Lung Dis 2017;21:1300-6. [Crossref] [PubMed]
21. Pan Y, Bai W, Chen J, et al. Diagnosing malignant pleural effusion using clinical and analytical parameters. J Clin Lab Anal 2019;33:e22689. [Crossref] [PubMed]
22. Elsammak MY, Attia A, Hassan HA, et al. Evaluation of pleural fluid human epididymis 4 (HE4) as a marker of malignant pleural effusion. Tumour Biol 2012;33:1701-7. [Crossref] [PubMed]
23. Altman DG, Bland JM. Diagnostic tests. 1: Sensitivity and specificity. BMJ 1994;308:1552. [Crossref] [PubMed]
24. Linnet K, Bossuyt PM, Moons KG, et al. Quantifying the Accuracy of a Diagnostic Test or Marker. Clin Chem 2012;58:1292-301. [Crossref] [PubMed]
25. Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ 2004;329:168-9. [Crossref] [PubMed]
26. Shi HZ, Liang QL, Jiang J, et al. Diagnostic value of carcinoembryonic antigen in malignant pleural effusion: a meta-analysis. Respirology 2008;13:518-27. [Crossref] [PubMed]
27. Nguyen AH, Miller EJ, Wichman CS, et al. Diagnostic value of tumor antigens in malignant pleural effusion: a meta-analysis. Transl Res 2015;166:432-9. [Crossref] [PubMed]
28. Wu Q, Li M, Zhang S, et al. Clinical diagnostic utility of CA 15-3 for the diagnosis of malignant pleural effusion: A meta-analysis. Exp Ther Med 2015;9:232-8. [Crossref] [PubMed]
29. Zhu J, Feng M, Liang L, et al. Is neuron-specific enolase useful for diagnosing malignant pleural effusions? evidence from a validation study and meta-analysis. BMC Cancer 2017;17:590. [Crossref] [PubMed]
30. Liang QL, Shi HZ, Qin XJ, et al. Diagnostic accuracy of tumour markers for malignant pleural effusion: a meta-analysis. Thorax 2008;63:35-41. [Crossref] [PubMed]