Cervical cancer occurs frequently in women and every two minutes, a woman dies because of cervical cancer worldwide (1,2). The introduction of cytology screening (Pap smear) has greatly reduced the mortality and morbidity rates for patients with invasive cervical cancers in the past decades. However, it has been demonstrated that the accuracy of Pap smear test ranged from 20% to 80%, and the results varied substantially in areas with different screening infrastructures, which therefore limited its efficacy for cervical cancer diagnosis (3-6). Persistently infected by human papillomavirus (HPV) with high risk type remains the major pathogens of cervical cancer, as a result, HPV DNA testing has been adopted for the triage of patients with atypical squamous cells of undetermined significance (2,4,7-9). Nevertheless, HPV triage yields a poor specificity, which diminishes the overall diagnostic value for cervical cancer screening (10,11). In this respect, discovering and developing new biomarkers which confer high sensitivity and specificity for cervical cancer detection is a matter of great urgency in the clinic. Epigenetic alterations, such as gene or DNA methylation, have been proposed as novel biomarkers for the cervical cancer detection. Among these altered and methylated genes, the paired boxed gene 1 (PAX1) and sex determining region Y-box 1 (SOX1) were both highlighted (12-26). Numerous studies have documented the promise of PAX1 or SOX1 DNA methylation in distinguishing patients with reactive atypia or CIN3+ (cervical intraepithelial neoplasia type III or worse) lesions from normal uterine cervixes (12-23). In order to make a comparison of the accuracy of DNA methylation and HPV DNA testing, we performed a comprehensive meta-analysis and evaluated the diagnostic performance of PAX1 and SOX1 methylation for cervical cancer screening.
This meta-analysis was conducted in compliance with the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) Statement issued in 2009 (27). The online PubMed and Chinese National Knowledge Infrastructure (CNKI) databases were searched up to August 31st 2015 for all published articles with no language restriction. Relevant publications were identified by searching for combinations of “PAX1 and/or SOX1 methylation”, “HPV” or “human papillomavirus”, “cervical cancer” or “cervical neoplasia” or “cervical carcinoma”, “high-grade squamous intraepithelial lesion” or “cervical intraepithelial neoplasia type III” or “CIN3”, and “diagnosis” or “sensitivity” or “specificity” or “AUC”.
The following criteria were used for the literature selection in this meta-analysis: (I) studies evaluated the diagnostic performance of PAX1 and/or SOX1 methylation or HPV DNA testing in cervical neoplasms; and (II) studies explicitly mentioned the sample size, sensitivity, specificity and their 95% confidence intervals (CIs) or other more detailed information. Literature were excluded according to the following criteria: (I) the control group and sample sizes were unclear; (II) studies without complete data including missing information of sensitivity, specificity or area under the curve (AUC) value, etc.; and (III) basic research, animal studies, meta analysis, review articles, letters, commentaries, abstracts presented at conferences, etc.
Data extraction and quality assessment
Two investigators independently judged study eligibility and extracted the relevant data from each article including the name of the first author, year of publication, country, patient size, control sources, sample types, CIN degrees, test method and the diagnostic results. In studies contained both a training and a validation group, data from each group was treated as a single study in the meta-analysis. Study quality was scored using a 14-item quality assessment of diagnostic accuracy studies (QUADAS) checklist and each was assessed as “Yes”, “Unclear” or “No”, corresponding to a score of “1”, “0” and “0”, respectively (28). Any disagreement was resolved by group consensus.
Statistical analysis was undertaken using Stata 12.0 (Stata Corporation, College Station, TX, USA), and Meta-disc 1.4 (XI Cochrane Colloquium, Barcelona, Spain) software. Diagnostic performance was assessed using a bivariate random-effects model in case of the existence of heterogeneity among studies (29). Heterogeneity from threshold and non-threshold effects were reflected by the Spearman correlation coefficient, Cochran’s-Q and I2 tests (30), respectively. Influence analysis and meta-regression were performed to trace potential sources of study heterogeneity. The covariates included the following: sample size (<100 or ≥100), cut-off value (PMR % <10 or ≥10), test method (QMSP or MS-HRM), and study quality (QUADAS ≥10 or <10). Deeks’ funnel plot asymmetry test was conducted to evaluate the potential publication bias, and significant level was set at P<0.05.
Study characteristics and quality
Figure 1 illustrates the articles search procedure: a total of 541 potentially relevant studies were identified from a primary literature search in electronic databases, and 498 studies were excluded due to the status that unrelated to PAX1, SOX1 methylation or cervical cancer diagnosis. The retrieved 43 studies received further evaluation: 2 of them were review articles and 28 were basic research which all failed to meet the inclusion criteria. In one study, the clinical accuracy calculated combined CIN3+ and endometrial complex hyperplasia was further excluded (21). Another study comprises normal and CIN2-3 samples in the paired controls was finally discarded as well (31). Therefore, 11 articles with 19 individual studies for PAX1 methylation, 10 for SOX1 methylation and 10 for HPV DNA testing were included in this meta-analysis. All of the 11 studies were conducted in Asia, including 7 studies in Taiwan and 4 in China mainland. The final diagnoses of all studies were determined by tissue-proven histopathology, and the evaluation method for DNA methylation comprises quantitative methylation-specific polymerase chain reaction (QMSP) and/or methylation-sensitive high-resolution melting (MS-HRM). The main features of each included study were described in Table 1. We evaluated the study quality of each included publications according to the 14 item QUADAS assessment tool (28). As shown in Figure 2, all of the 11 studies got QUADAS scores more than 8 and revealed lower risks of bias, suggesting a relatively high quality of the included studies.
Heterogeneity from threshold and non-threshold effects were assessed using Meta-disc 1.4 software. As shown in Table 2, Spearman correlation coefficient in PAX1 methylation test presented a P value of 0.181, indicating that there was no heterogeneity from threshold effect. Additionally, the Cochran’s-Q test yielded a Q value of 30.70 (P>0.01), with I2<50%, suggesting that non-threshold effect is not likely to be a source of heterogeneity. Nevertheless, heterogeneity generated by threshold or non-threshold effects seemed to exist in other pooled analyses (Table S1).
As indicated in Table 2, the pooled accuracies for PAX1 and/or SOX1 methylation, alone, in parallel or in sequential combinations, were determined to assess their usefulness as biomarkers for screening of patients with HSIL and/or cervical carcinoma. The pooled sensitivity, specificity, DOR and AUC for PAX1 methylation test were 0.73 (95% CI: 0.70–0.75), 0.87 (95% CI: 0.85–0.89), 27.51 (95% CI: 17.24–43.88) and 0.91, respectively, versus those of 0.71 (95% CI: 0.67–0.74), 0.64 (95% CI: 0.61–0.67), 8.73 (95% CI: 5.75–13.27) and 0.82 for SOX1 methylation. The forest plots of pooled sensitivity, specificity and summary receiver operator characteristic (SROC) curves for PAX1 and SOX1 methylation are displayed in Figures 3,4. For the HPV DNA testing, it yielded an AUC value of 0.77, with pooled sensitivity of 0.81 (95% CI: 0.77–0.85) and specificity of 0.70 (95% CI: 0.67–0.72). Forest plots of pooled sensitivity, specificity as well as SROC curve for HPV DNA are shown in Figure S1.
A random effect model was applied in the stratified meta-analyses due to the existence of significant heterogeneities among studies. The results for the stratified analyses were listed in Table 2. The paralleled PAX1/HPV, SOX1/HPV and PAX1/SOX1 tests achieved AUC values of 0.89, 0.84 and 0.89, respectively, under which, the pooled sensitivity were 0.75 (95% CI: 0.73–0.78), 0.64 (95% CI: 0.58–0.69) and 0.72 (95% CI: 0.69–0.74), respectively; the pooled specificity were 0.81 (95% CI: 0.79–0.82), 0.48 (95% CI: 0.45–0.52) and 0.77 (95% CI: 0.76–0.79), respectively.
Influence assay and meta-regression
We performed influence analysis based on the platform of Stata 12.0 software. As shown in Figure 5A, influence analysis and outlier detection identified 2 outlier studies for PAX1 methylation test. After adjustment of the outlier studies, the overall pooled sensitivity increased from 0.73 to 0.74, PLR decreased from 5.80 to 5.59, NLR decreased from 0.30 to 0.27, DOR elevated from 27.51 to 29.84. Moreover, the P value of spearman correlation coefficient from threshold effect increased from 0.459 to 0.625, and P value from Cochran’s-Q test was attenuated form 0.03 to 0.05, and I2 declined from 41.4% to 38.3%, hinting that the outlier is likely a source of heterogeneity. For the HPV DNA test, we identified 2 outlier studies as well, and the data revealed that the outliers also contributed to the heterogeneity sources (Figure S2). No outlier studies were identified in SOX1 methylation test (Figure 5B). We further conducted meta-regression analysis based on 5 pre-specified covariates: sample size (<100 or ≥100), cut-off value (PMR % <10 or ≥10), test method (QMSP or MS-HRM), and study quality (QUADAS ≥10 or <10). However, these covariates showed a low likelihood of sources of inter-study heterogeneity (Table 3).
The funnel plots for publication bias showed no asymmetry for the pooled PAX1 and SOX1 methylation analysis, corresponding to a P value of 0.645 and 0.651, respectively, indicating that there was no publication bias in the meta-analysis (Figure 6). Additionally, the P value of the Deeks’ test for HPV DNA was estimated to be 0.869, also showed a low likelihood of publication bias (Figure S3).
Infection with oncogenic types of HPV is the most significant risk factor in the etiology of cervical cancer, and is present in nearly all cervical cancers (2,4,8,32,33). Molecular tests detecting HPV DNA are now being available either commercially or in-house developed. The sensitivity of HPV DNA testing is satisfactory, whereas the high prevalence of transient HPV infections had limited the specificity of this approach (10,11). Currently, there are several test methods for HPV DNA assay, such as PCR, hybrid capture II (HC2), strip test (for E6 detection) as well as other novel techniques (5,12-15,34,35), Among these evaluation tools, PCR and HC2 tests remain the most commonly used techniques (34,35). Notwithstanding, single HPV DNA testing using PCR or HC2 method seems to yield inconstant results for the detection of cervical precancerous lesions (CIN2 or worse). For instance, previous studies had documented that the sensitivity of HPV DNA testing using PCR method ranged from 0.66 to 0.89, whereas the specificity was from 0.64 to 0.68 (12,15). However, in a published meta-analysis study, the result showed that PCR test achieved a pooled sensitivity of 0.809 and specificity of 0.947 (5). As reported, HPV DNA test utilizing HC2 method may confer a relatively higher accuracy than using PCR analysis. In the aforementioned meta-analysis, the HC2 method displayed a pooled sensitivity of 0.90 and specificity of 0.865 in primary cervical screening (5). Nevertheless, in Wang’s study, the accuracy of HC2 test in triage of high-grade squamous intraepithelial lesions was unsatisfactory, in which the sensitivity and specificity were only 0.677 and 0.545, respectively (13). Despite the reported inconstant results for these methods, these techniques often lack standardization and comparability (34,35). It appears therefore that single HPV DNA testing is not recommended for screening purposes (10,11).
For early cervical cancer diagnosis, DNA methylation of the PAX1 or SOX1 gene, has been proposed as new biomarkers for the cancer screening against HPV DNA testing (12-26). As the potential diagnostic value of DNA methylation for cervical cancer screening has not yet been well elucidated thus far, this study systematically evaluated and compared the diagnostic accuracy of PAX1 and SOX1 methylation and HPV DNA testing in the diagnosis of HSIL or CIN3+ lesions or cervical cancer. As shown in our data, PAX1 methylation achieved a higher pooled diagnostic accuracy when compared with SOX1 methylation or HPV DNA testing. The pooled sensitivity and specificity of PAX1 methylation was 0.73 and 0.87, respectively. Although the pooled sensitivity appeared not very high, the ROC AUC was 0.91, suggesting an overall high accuracy of this diagnostic test. Diagnostic odds ratio (DOR) is one of the key indicators in assessing the accuracy of one diagnostic test, and that a DOR smaller than 1.0 often suggests a low discriminating value for a diagnostic test (36,37). Importantly, the pooled DOR for PAX1 methylation was 27.51, indicating a better discriminatory test performance of PAX1 methylation for cervical cancer detection. Moreover, the pooled PLR of 5.80 and NLR of 0.30, also suggested that patients with HSIL or CIN3+ or cervical cancer had nearly 6 fold higher chance of being PAX1 methylation test positive than other normal individuals, with a ratio of 30% likely to be false-negative in a negative result of the methylation test, revealing a relatively high power in ruling out high risk of cervical cancer.
As a comparison, SOX1 methylation test revealed an overall accuracy inferior to that of PAX1 methylation: despite a reduction in the sensitivity and specificity, other parameters were lower than PAX1 methylation as well. Yet, the HPV DNA testing harvested the highest combined sensitivity of 0.83 among these assays, but the pooled specificity was estimated to be 0.67. Besides that, other indices like pooled PLR, NLR and DOR, also indicated an overall diagnostic accuracy inferior to PAX1 methylation test. A published research suggested that the methylation test of PAX1 gene harbored higher diagnostic accuracy against HPV DNA testing in detecting cervical cancer (20). Similarly, other studies also documented that quantitative measurement of PAX1 hypermethylation in cervical scrapings is highly accurate than both SOX1 hypermethylation and HPV DNA assay, which the sensitivity was 0.86, and specificity was 0.85 under the ROC curve (12,17). Our results agree well with these data.
In order to verify whether the combing parallel testing of DNA methylation and HPV DNA could benefit the diagnostic accuracy, we further conducted the stratified analyses. As expected, the results displayed that PAX1/HPV paralleled test harbored a high AUC value of 0.89, but both pooled sensitivity and specificity decreased when compared with that of PAX1 methylation alone. However, the overall pooled diagnostic accuracy for PAX1/HPV test was comparable to that of PAX1 methylation alone and better than single HPV DNA testing. For the paralleled SOX1/HPV test, although their AUC and DOR were both elevated as compared to that of SOX1 methylation alone, the pooled sensitivity, specificity as well as other indices was discounted. In addition, we found the parallel testing of PAX1 and SOX1 methylation harvested a sensitivity of 0.72, specificity of 0.77 and DOR of 24.41 with an AUC value of 0.89, suggesting that a combined detection of PAX1 and SOX1 methylation harbored a moderate accuracy for cervical cancer detection. Evidence from a recent study suggested that PAX1 methylation harbored a better diagnostic performance with a specificity of 0.88 when combined with sequential testing of HPV (14). In addition, the paneled assay of PAX1/SOX1 methylation has been previously described as well, which the sensitivity was greater than 0.70, and specificity greater than 0.90 (15). For the combined PAX1 and HPV DNA testing, the published data had documented that when combining with HPV DNA 16/18 typing, the PAX1 testing showed a sensitivity of almost 0.90 and a specificity of 0.83 (17). Consistent with these results, our data showed promising accuracy for the paralleled PAX1/HPV test.
In this study, heterogeneity from threshold and non-threshold effects existed in the pooled studies. For its causes, the uniformed cut-off value setting may contribute to the heterogeneity of threshold effect. On the other hand, the non-threshold effect is another important source of heterogeneity, and parameters of Cochran’s-Q and I2 tests from pooled DOR are good indicators in discussing heterogeneity from nonthreshold effects (36). In this meta-analysis, the forest plots of DOR from each study (except PAX1, SOX1 and SOX1/HPV tests) did not distribute along a straight line (not shown), corresponding to the P values in Cochran’s-Q test less than 0.01 and I2 more than 50%. It is speculated that different measurement methods, or sample types may contribute to the heterogeneity sources. For instance, two different test methods, QMSP and HRM were used in measuring DNA methylation in our study. In addition to the test method, the reference method was also different among studies. In consequence, we further conducted influence and meta-regression analyses and our results revealed that the outlier studies were likely to be a source of heterogeneity.
This meta-analysis demonstrates that PAX1 methylation achieves a promising diagnostic performance for cervical cancer detection, and parallel testing of PAX1 methylation and HPV DNA confers an improved diagnostic accuracy than HPV DNA testing alone. For the SOX1 methylation test, it yields a moderate accuracy in diagnosing cervical neoplasia. Hence, PAX1 or SOX1 methylation could potentially be treated as an auxiliary biomarker for cervical cancer screening. Further high quality studies are still warranted to confirm our analyses.
This study was supported by the National Clinical Key Specialty Construction Program of China.
Conflicts of Interest: The authors have no conflicts of interest to declare.
- Forouzanfar MH, Foreman KJ, Delossantos AM, et al. Breast and cervical cancer in 187 countries between 1980 and 2010: a systematic analysis. Lancet 2011;378:1461-84. [Crossref] [PubMed]
- Köse FM, Naki MM. Cervical premalignant lesions and their management. J Turk Ger Gynecol Assoc 2014;15:109-21. [Crossref] [PubMed]
- Austin RM, Zhao C. Is 58% sensitivity for detection of cervical intraepithelial neoplasia 3 and invasive cervical cancer optimal for cervical screening? Cytojournal 2014;11:14. [Crossref] [PubMed]
- Mayrand MH, Duarte-Franco E, Rodrigues I, et al. Human papillomavirus DNA versus Papanicolaou screening tests for cervical cancer. N Engl J Med 2007;357:1579-88. [Crossref] [PubMed]
- Koliopoulos G, Arbyn M, Martin-Hirsch P, et al. Diagnostic accuracy of human papillomavirus testing in primary cervical screening: a systematic review and meta-analysis of non-randomized studies. Gynecol Oncol 2007;104:232-46. [Crossref] [PubMed]
- Nanda K, McCrory DC, Myers ER, et al. Accuracy of the Papanicolaou test in screening for and follow-up of cervical cytologic abnormalities: a systematic review. Ann Intern Med 2000;132:810-9. [Crossref] [PubMed]
- Arbyn M, Buntinx F, Van Ranst M, et al. Virologic versus cytologic triage of women with equivocal Pap smears: a meta-analysis of the accuracy to detect high-grade intraepithelial neoplasia. J Natl Cancer Inst 2004;96:280-93. [Crossref] [PubMed]
- Sankaranarayanan R, Nene BM, Shastri SS, et al. HPV screening for cervical cancer in rural India. N Engl J Med 2009;360:1385-94. [Crossref] [PubMed]
- Mandelblatt JS, Lawrence WF, Womack SM, et al. Benefits and costs of using HPV testing to screen for cervical cancer. JAMA 2002;287:2372-81. [Crossref] [PubMed]
- Dehn D, Torkko KC, Shroyer KR. Human papillomavirus testing and molecular markers of cervical dysplasia and carcinoma. Cancer 2007;111:1-14. [Crossref] [PubMed]
- Woodman CB, Collins SI, Young LS. The natural history of cervical HPV infection: unresolved issues. Nat Rev Cancer 2007;7:11-22. [Crossref] [PubMed]
- Lai HC, Lin YW, Huang TH, et al. Identification of novel DNA methylation markers in cervical cancer. Int J Cancer 2008;123:161-7. [Crossref] [PubMed]
- Wang ZM. PAX1 methylation analysis by MS-HRM is useful in triage of high-grade squamous intraepithelial lesions. Asian Pac J Cancer Prev 2014;15:891-4. [Crossref] [PubMed]
- Lai HC, Ou YC, Chen TC, et al. PAX1/SOX1 DNA methylation and cervical neoplasia detection: a Taiwanese Gynecologic Oncology Group (TGOG) study. Cancer Med 2014;3:1062-74. [Crossref] [PubMed]
- Lai HC, Lin YW, Huang RL, et al. Quantitative DNA methylation analysis detects cervical intraepithelial neoplasms type 3 and worse. Cancer 2010;116:4266-74. [Crossref] [PubMed]
- Chao TK, Ke FY, Liao YP, et al. Triage of cervical cytological diagnoses of atypical squamous cells by DNA methylation of paired boxed gene 1 (PAX1). Diagn Cytopathol 2013;41:41-6. [Crossref] [PubMed]
- Kan YY, Liou YL, Wang HJ, et al. PAX1 methylation as a potential biomarker for cervical cancer screening. Int J Gynecol Cancer 2014;24:928-34. [Crossref] [PubMed]
- Zhou AF, Li SR, Wang ZM. The Role of DNA Hypemethylation of PAX1 in early diagnosis of cervical cancer and precursor lesions. Acta Acad Med Weifang 2011;33:439-41.
- Xu ZM, Qin SX, Pei F, et al. Quantitative methylation-specific PCR for PAX1 gene and its significance in detection of cervical cancer. Chinese Clinical Oncology 2013;18:10-4.
- Huang TH, Lai HC, Liu HW, et al. Quantitative analysis of methylation status of the PAX1 gene for detection of cervical cancer. Int J Gynecol Cancer 2010;20:513-9. [Crossref] [PubMed]
- Chang CC, Ou YC, Wang KL, et al. Triage of Atypical Glandular Cell by SOX1 and POU4F3 Methylation: A Taiwanese Gynecologic Oncology Group (TGOG) Study. PLoS One 2015;10:e0128705. [Crossref] [PubMed]
- Chang CC, Huang RL, Liao YP, et al. Concordance analysis of methylation biomarkers detection in self-collected and physician-collected samples in cervical neoplasm. BMC Cancer 2015;15:418. [Crossref] [PubMed]
- Li SR, Wang ZM, Wang YH, et al. Value of PAX1 Methylation Analysis by MS-HRM in the Triage of Atypical Squamous Cells of Undetermined Significance. Asian Pac J Cancer Prev 2015;16:5843-6. [Crossref] [PubMed]
- Lim EH, Ng SL, Li JL, et al. Cervical dysplasia: assessing methylation status (Methylight) of CCNA1, DAPK1, HS3ST2, PAX1 and TFPI2 to improve diagnostic accuracy. Gynecol Oncol 2010;119:225-31. [Crossref] [PubMed]
- Vasiljević N, Scibior-Bentkowska D, Brentnall AR, et al. Credentialing of DNA methylation assays for human genes as diagnostic biomarkers of cervical intraepithelial neoplasia in high-risk HPV positive women. Gynecol Oncol 2014;132:709-14. [Crossref] [PubMed]
- Chang CC, Huang RL, Wang HC, et al. High methylation rate of LMX1A, NKX6-1, PAX1, PTPRR, SOX1, and ZNF582 genes in cervical adenocarcinoma. Int J Gynecol Cancer 2014;24:201-9. [Crossref] [PubMed]
- Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009;62:1006-12. [Crossref] [PubMed]
- Whiting P, Rutjes AW, Reitsma JB, et al. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003;3:25. [Crossref] [PubMed]
- Vamvakas EC. Meta-analyses of studies of the diagnostic accuracy of laboratory tests: a review of the concepts and methods. Arch Pathol Lab Med 1998;122:675-86. [PubMed]
- Zamora J, Abraira V, Muriel A, et al. Meta-DiSc: a software for meta-analysis of test accuracy data. BMC Med Res Methodol 2006;6:31. [Crossref] [PubMed]
- Xu J, Xu L, Yang B, et al. Assessing methylation status of PAX1 in cervical scrapings, as a novel diagnostic and predictive biomarker, was closely related to screen cervical cancer. Int J Clin Exp Pathol 2015;8:1674-81. [PubMed]
- Dunne EF, Park IU. HPV and HPV-associated diseases. Infect Dis Clin North Am 2013;27:765-78. [Crossref] [PubMed]
- Priebe AM. 2012 cervical cancer screening guidelines and the future role of HPV testing. Clin Obstet Gynecol 2013;56:44-50. [Crossref] [PubMed]
- Agorastos T, Sotiriadis A, Chatzigeorgiou K. Can HPV testing replace the pap smear? Ann N Y Acad Sci 2010;1205:51-6. [Crossref] [PubMed]
- Kroupis C, Vourlidis N. Human papilloma virus (HPV) molecular diagnostics. Clin Chem Lab Med 2011;49:1783-99. [Crossref] [PubMed]
- Glas AS, Lijmer JG, Prins MH, et al. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol 2003;56:1129-35. [Crossref] [PubMed]
- Shen Y, Wang T, Yang T, et al. Diagnostic value of circulating microRNAs for lung cancer: a meta-analysis. Genet Test Mol Biomarkers 2013;17:359-66. [Crossref] [PubMed]