The diagnostic performance of the Gynecologic Imaging Reporting and Data System (GI-RADS) in adnexal masses

Introduction

Adnexal masses are common in the female genital system, and most of them are benign. Nevertheless, ovarian cancer is the third most common gynecologic malignancy (1.6%) and the leading cause of death among women with gynecologic cancer (1,2). Therefore, it is significant to find early and accurate diagnostic methods to recognize the malignant adnexal masses. Now, as recommended by the guidelines of ovarian cancer, differential diagnostic methods include ultrasound, Computerized Tomography (CT), Magnetic Resonance Imaging (MRI), and serum cancer antigen tests (3-5). Ultrasound examination is used in investigating pelvic tumors. However, because of the experiential discrepancy among inspectors, the results of evaluating adnexal masses are not objective enough. According to a multicentre clinical trial, the false positive rate of investigating ovarian cancer is 24% (6). Meanwhile, problems also exist in the interpretation of the information from ultrasound doctors to clinicians (6).

Similar to Breast Imaging Reporting and Data System (BI-RADS), designed by the American College of Radiology to reduce the influence of subjective judgment on ultrasonic examination results of breasts, Gynecologic Imaging Reporting and Data System (GI-RADS) was introduced by Amor. F to standardize the ultrasound report of the gynecologic system (7,8). This system included several ultrasonic morphological and clinical features and they were laterality, maximum diameter, echogenicity, wall thickness, cystic content, solid areas, septations, solid papillary projections, the presence of ascites, the doppler evaluation. The classification of GI-RADS includes GI-RADS 1 to GI-RADS 5, and a higher level indicates that the mass is more likely to be malignant. The kappa coefficients of GI-RADS categorization were 0.896 (95% CI: 0.775 to 1.000), according to a previous study (9). The diagnostic performance of GI-RADS in different studies is highly heterogeneous, and no clinical trial of large samples had investigated such discrepancy. Thus, this present study aimed to design a meta-analysis to evaluate and review the effect of the GI-RADS. Further, we also compared different ovarian cancer diagnostic methods with GI-RADS. We presented this article following the PRISMA reporting checklist (available at http://dx.doi.org/10.21037/atm-20-5170).

Methods

Search strategy

We searched the studies published from 1990 to February 2020 in PubMed, MEDLINE, and EMBASE, and the search terms were “Gynecologic Imaging Reporting and Data System” and “GI-RADS”. After reading the title and abstract of the studies, three reviewers (W Guo, Xiuhe Zou and HY Xu) decided whether the studies met the inclusion criteria independently. One experienced investigator (XL Ma) assessed the quality of the included studies and resolved the conflicts between three reviewers.

Inclusion and exclusion criteria

Inclusion criteria were that: (I) the patients had adnexal masses; (II) the adnexal mass had been verified by golden standards (biopsy or follow-up), and the minimal required follow-up time was 6 weeks; (III) the adnexal mass was evaluated by ultrasound and classified by the GI-RADS only; (IV) the article was written in English. The exclusion criteria were that: (I) studies only had abstracts or lacked sufficient data of the diagnostic effect; (II) the adnexal mass was evaluated by other imaging methods or methods combined with ultrasound; (III) the langue of articles was not English.

Data extraction

Two reviewers extracted these data: study characteristics, demographic characteristics, and outcomes from the selected articles. The details are as followings:

• Study characteristics: author, year of the study, country, amount of the patients and masses, ultrasound equipment, probe frequency, the technology of the ultrasound, and the golden standard;
• Demographic characteristics: age, the prevalence rate for adnexal malignancy, premenopausal rate;
• Outcomes: specificity, sensitivity, positive predictive value, negative predictive value, and value of true positive (TP), false positive (FP), true negative (TN), false negative (FN) for the ability of GI-RADS to recognize the adnexal malignancy (10).

Statistical analysis

We used Meta-Disc version 1.4 statistical software to calculate the pooled specificity, sensitivity, positive likelihood ratios (PLR), negative likelihood ratios (NLR), and diagnostic odds ratio (DOR) with 95% confidence intervals (CIs). Also, the summary receiver operator characteristics (SROC) was constructed by this software to evaluate the relationship between specificity and sensitivity. We also used the inconsistency index (I²) to detect heterogeneity among the included studies and would use the random effect model in the meta-analysis if I² was more than 50%. The Deeks’ funnel plot asymmetry test performed in Stata version 16.0 software (StatsCorp., College Station, TX) was used to value publication bias.

Quality assessment

To assess the included articles, we used the Quality Assessment of Diagnostic Accuracy Studies version 2 (QUADAS-2). The QUADAS-2 can evaluate the quality of studies from 14 items, four domains, and two aspects (11). According to the assessment results, these ten articles were reliable enough to conclude the ability of GI-RADS to differentiate adnexal masses. The QUADAS-2 was operated in Review Manager version 5.3 (Cochrane Informatics and Knowledge Management Department, Copenhagen, Denmark).

Results

Study selection

After removing the duplicates, we included 311 articles by the initial publication search and selected 77 studies for further filtration by reading the full text. Then, 67 of these publications failing to meet the inclusion criteria were excluded: 60 articles were not related to the subject; 4 articles did not use the GI-RADS to evaluate the adnexal masses; 3 articles did not use the ultrasound to detect the masses.

Additionally, though written by the same authors, two studies of different sample sizes were included.

Finally, we included a total of 10 articles in this study, and the selection flow chart was shown in Figure 1.

Figure 1 Flow chart of researches selection.

Eligible study characteristics

Ten selected studies were published from 2009 to 2019, and the details of these studies were shown in Table 1. These studies were conducted in 7 different countries. The total number of patients was 2,474, and that of masses was 2,723. The mean age of all the patients was 42.75 (age range, 6–89), and the prevalence rate of malignant tumors ranged from 13.45% to 41.83%. The type of the ultrasound equipment in these studies was Voluson 730 (GE Healthcare, Milwaukee, WI, USA), GE E8 EXPERT (GE Medical Systems, Zipf, Austria), GE S8 (GE Medical Systems, Zipf, Austria), iU22 digital scanner (Philips Healthcare, Bothell, WA, USA), Voluson E8 (GE Medical Systems, Zipf, Austria), SonoScape S40 Exp/S40 Pro/S40/S35 (Sonoscape Medical Hong Kong Co., Ltd), DC-7 Mindary (DC-7 Mindary Medical Ltd, China), ACUSON X150 ultrasound system (Siemens Health care, Mountain View, CA, USA), ProSound Alpha 7 ultrasound (Hitachi Aloka Medical America, Inc. Germany). The golden standard of the differentiation of the gynecologic malignancy was biopsy and clinical symptoms during the follow-up. Histology diagnosis was used as the golden standard of the differentiation of the gynecologic malignancy in all included studies. What’s more, in Tan Z, Basha MAA, and Behnamfar F’studies, some of the patients had a spontaneous resolution in followed up by ultrasound study. Behnamfar F’s study defined the follow-up interval was 6 weeks, and the other two studies didn’t mention the specific follow-up time.

Table 1 Basic characteristics of included studies
Full table

Diagnostic value of differentiating malignancy

The pooled sensitivity of selected studies was 0.95 (95% CI: 0.94 to 0.97, I²=61.5%), and the pooled specificity was 0.86 (95% CI: 0.84 to 0.88, I²=94.3%). The pooled NLR and PLR were 0.06 (95% CI: 0.04 to 0.10, I²=59.3%), and 8.30 (95% CI: 4.93 to 13.97, I²=94.2%), respectively. The pooled DOR for GI-RADS was 174.59 (95% CI: 76.70 to 397.42, I²=74.6%). The SROC was shown in Figure 2, and the overall area under the curve (AUC) was 0.9806 (SE, 0.0160).

Figure 2 Plots of sensitivity (A), specificity (B), negative likelihood ratio (C), positive likelihood ratio (D), diagnostic odds ratio (E) and receiver operating characteristic curve (F) of diagnosing adnexal mass. Positive LR, positive likelihood ratio; negative LR, negative likelihood ratio; AUC, area under the curve; CI, confidence intervals; diagnostic OR, diagnostic odds ratio; SROC, summary receiver operator characteristics.

Assessment of publication bias

The Deeks’ funnel plot asymmetry test was used to evaluate the publication bias in the included studies (Figure 3). The results showed that the diagnostic OR of GI-RADS was not significantly biased in the differentiation of benign and malignant adnexal masses (P=0.6).

Figure 3 Risk of bias and applicability concerns: (A) graph; (B) summary.

Quality assessment

The exact assessment results of the eligible studies were shown in Figure 4, and the quality of these studies was sufficient for analysis.

Figure 4 Deeks’ funnel plot summarizes the publication bias in our meta-analysis of the diagnostic odds ratios.

Discussion

This meta-analysis included ten studies from 7 countries to evaluate the diagnostic value of the GI-RADS for adnexal masses. Notably, Amor F. conducted two studies with different patients in 2009 and 2011. The diagnostic accuracy of ultrasound depended on the operators’ experience, and the difference rate between experts and routines was 15% (95% CI: 0–29%) (8,21-23). Thus, introducing the GI-RADS can strengthen the consistency among the different examiners and improve the diagnostic ability to differentiate the malignancy from the adnexal masses.

Our research systematically analyzed the data of the diagnostic performance of the GI-RADS in these ten studies. The pooled sensitivity and specificity were 0.95 and 0.86, respectively. The pooled NLR was 0.06, and PLR was 8.30. PLR >10 and NLR <0.1 mean high diagnostic accuracy (24). Although the pooled NLR was 0.06 and PLR 8.30 was in our study, the PLR value failed to draw a convincing conclusion of the presence of the adnexal malignancy. Furthermore, the diagnostic accuracy of GI-RADS evaluated by the AUC was 0.9806, indicating a high diagnostic value (25). All of these results showed that the GI-RADS could differentiate gynecological adnexal malignancy from adnexal masses.

There were several limitations to this study. Firstly, since I² of the pooled sensitivity and specificity were over 50%, we should consider the heterogeneity of data. However, lacking enough data in the included articles, we could not perform a sub-analysis. Besides, various types of ultrasound equipment used in different studies might account for heterogeneity among studies. Also, the age range of patients, the prevalence rate for adnexal malignancy, and menopausal proportions varied in these studies, which might increase the heterogeneity of the result. Finally, there were no factors in the included articles that allowed for subgroup analysis, which might hinder finding subgroups with higher diagnostic efficacy. Therefore, different ultrasound parameters are desired in subsequent studies to explore better diagnostic performance.

Although we excluded studies using other examine methods such as MRI and contrast-enhanced ultrasound (CEUS), their diagnostic power is worth further discussion. A previous study showed that when combining GI-RADS with 3D-CEUS, the diagnostic sensitivity, specificity, AUC, and kappa value were 94.2%, 95.5%, 0.969, and 0.872, respectively (26). Compared with using GI-RADS alone, the combination of GI-RADS and 3D-CEUS had higher specificity and lower sensitivity and smaller AUC according to this research, partly because the 3D-CEUS could not show real-time hemodynamic variance in the masses. Due to the limited number of studies included, large-scale and multicentre studies are required to compare these two methods.

Furthermore, the detectable rate of malignant adnexal masses detected by the ultrasound was low (8–20%), so MRI was introduced to a reporting data system named Ovarian Adnexal Reporting Data System Magnetic Resonance Imaging (O-RADS MRI). Based on a multicentre cohort study, the sensitivity, specificity, and AUC of O-RADS MRI were 0.93 (95% CI: 0.89–0.96), 0.91 (95% CI: 0.89–0.93), and 0.961 (95% CI: 0.948–0.971) (27-29). The diagnostic performance and accuracy of MRI were similar to that of GI-RADS.

In conclusion, GI-RADS is useful and reliable to improve the diagnostic ability of ultrasound to differentiate adnexal masses and malignant tumors. It is an effective method to settle the inconsistency among different ultrasound operators. With the development of the novel diagnostic tool, combining the GI-RADS with other examinations is a trend. However, further studies are required to confirm the results of our research.

Acknowledgments

We would like to show our deepest thanks to Dr. Zikuan Song and Dr. Yinan Xiao, who have helped us to revise our manuscript.

Funding: None.

Footnote

Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at http://dx.doi.org/10.21037/atm-20-5170

Peer Review File: Available at http://dx.doi.org/10.21037/atm-20-5170

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm-20-5170). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Bray F, Ferlay J, Soerjomataram I, et al. Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2018;68:394-424. [Crossref] [PubMed]
2. Coleman RL, Spirtos NM, Enserro D, et al. Secondary surgical cytoreduction for recurrent ovarian cancer. N Engl J Med 2019;381:1929-39. [Crossref] [PubMed]
3. Biggs WS, Marks ST. Diagnosis and management of adnexal masses. Am Fam Physician 2016;93:676-81. [PubMed]
4. Armstrong DK, Alvarez RD, Bakkum-Gamez JN, et al. NCCN guidelines insights: Ovarian cancer, version 1. J Natl Compr Canc Netw 2019;17:896-909. [Crossref] [PubMed]
5. Jelovac D, Armstrong DK. Recent progress in the diagnosis and treatment of ovarian cancer. CA Cancer J Clin 2011;61:183-203. [Crossref] [PubMed]
6. Yazbek J, Raju SK, Ben-Nagi J, et al. Effect of quality of gynaecological ultrasonography on management of patients with suspected ovarian cancer: A randomised controlled trial. Lancet Oncol 2008;9:124-31. [Crossref] [PubMed]
7. Starren J, Johnson SM. Expressiveness of the breast imaging reporting and database system (bi-rads). Proc AMIA Annu Fall Symp 1997.655-9. [PubMed]
8. Amor F, Vaccaro H, Alcázar JL, et al. Gynecologic imaging reporting and data system: A new proposal for classifying adnexal masses on the basis of sonographic findings. J Ultrasound Med 2009;28:285-91. [Crossref] [PubMed]
9. Basha MAA, Refaat R, Ibrahim SA, et al. Gynecology imaging reporting and data system (gi-rads): Diagnostic performance and inter-reviewer agreement. Eur Radiol 2019;29:5981-90. [Crossref] [PubMed]
10. Ma X, Zeng H, Zhang J, et al. New practical methods to obtain crucial data in performing diagnostic meta-analysis of the published literature. J Evid Based Med 2018;11:56-63. [Crossref] [PubMed]
11. Hollingworth W, Medina LS, Lenkinski RE, et al. Interrater reliability in assessing quality of diagnostic accuracy studies using the quadas tool. A preliminary assessment. Acad Radiol 2006;13:803-10. [Crossref] [PubMed]
12. Amor F, Alcázar JL, Vaccaro H, et al. Gi-rads reporting system for ultrasound evaluation of adnexal masses in clinical practice: A prospective multicenter study. Ultrasound Obstet Gynecol 2011;38:450-5. [Crossref] [PubMed]
13. Diaz L, Zambrano B, Adami FJ, et al. External validation of gynecological imaging and reporting data system for sonographic evaluation of adnexal masses. Donald School Journal of Ultrasound in Obstetrics and Gynecology 2017;11:135-40. [Crossref]
14. Koneczny J, Czekierdowski A, Florczak M, et al. The use of sonographic subjective tumor assessment, iota logistic regression model 1, iota simple rules and gi-rads system in the preoperative prediction of malignancy in women with adnexal masses. Ginekol Pol 2017;88:647-53. [Crossref] [PubMed]
15. Zhang T, Li F, Liu J, et al. Diagnostic performance of the gynecology imaging reporting and data system for malignant adnexal masses. Int J Gynaecol Obstet 2017;137:325-31. [Crossref] [PubMed]
16. Migda M, Bartosz M, Migda MS, et al. Diagnostic value of the gynecology imaging reporting and data system (gi-rads) with the ovarian malignancy marker ca-125 in preoperative adnexal tumor assessment. J Ovarian Res 2018;11:92. [Crossref] [PubMed]
17. Basha MAA, Refaat R, Ibrahim SA, et al. Gynecology imaging reporting and data system (gi-rads): Diagnostic performance and inter-reviewer agreement. Eur Radiol 2019;29:5981-90. [Crossref] [PubMed]
18. Behnamfar F, Adibi A, Khadra H, et al. Diagnostic accuracy of gynecology imaging reporting and data system in evaluation of adnexal lesions. J Res Med Sci 2019;24:57. [Crossref] [PubMed]
19. Zheng H, Tie Y, Wang X, et al. Assessment of the diagnostic value of using serum ca125 and girads system in the evaluation of adnexal masses. Medicine (Baltimore) 2019;98:e14577. [Crossref] [PubMed]
20. Khalaf LMR, Desoky HHM, Seifeldein GS, et al. The diagnostic efficacy of gynecology imaging reporting and data system (gi-rads): Single-center prospective cross-sectional study. Egyptian J Radiol Nuclear Med 2019;50:61. [Crossref]
21. Andreotti RF, Timmerman D, Benacerraf BR, et al. Ovarian-adnexal reporting lexicon for ultrasound: A white paper of the acr ovarian-adnexal reporting and data system committee. J Am Coll Radiol 2018;15:1415-29. [Crossref] [PubMed]
22. Suh-Burgmann EJ, Flanagan T, Lee N, et al. Large-scale implementation of structured reporting of adnexal masses on ultrasound. J Am Coll Radiol 2018;15:755-61. [Crossref] [PubMed]
23. Suh-Burgmann E, Flanagan T, Osinski T, et al. Prospective validation of a standardized ultrasonography-based ovarian cancer risk assessment system. Obstet Gynecol 2018;132:1101-1. [Crossref] [PubMed]
24. Deeks JJ. Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. BMJ 2001;323:157-62. [Crossref] [PubMed]
25. Metz CE. Basic principles of roc analysis. Semin Nucl Med 1978;8:283-98. [Crossref] [PubMed]
26. Wang X, Yang S, Lv G, et al. Combination of gi-rads and 3d-ceus for differential diagnosis of ovarian masses. Rev Assoc Med Bras 2019;65:959-64. [Crossref] [PubMed]
27. Thomassin-Naggara I, Poncelet E, Jalaguier-Coudray A, et al. Ovarian-adnexal reporting data system magnetic resonance imaging (o-rads mri) score for risk stratification of sonographically indeterminate adnexal masses. JAMA Netw Open 2020;3:e1919896. [Crossref] [PubMed]
28. Andreotti RF, Timmerman D, Strachowski LM, et al. O-rads us risk stratification and management system: A consensus guideline from the acr ovarian-adnexal reporting and data system committee. Radiology 2020;294:168-85. [Crossref] [PubMed]
29. Roth CG, Marzio DHD, Guglielmo FF. Contributions of magnetic resonance imaging to gastroenterological practice: Mris for gis. Dig Dis Sci 2018;63:1102-22. [Crossref] [PubMed]