Artificial intelligence-tutoring problem-based learning in ophthalmology clerkship

Introduction

The development of artificial intelligence (AI) provides new tools to solve problems in various medical fields (1). AI algorithms have obvious advantages in identifying and extracting image features (2). Therefore, AI models have been shown to performed equally with or better than human doctors in disease diagnosis based on signs or morphologies, such as skin cancer (3), diabetic retinopathy (4-6), congenital cataract (7), and pathological section identification (8-10). However, AI’s potential of aiding medical teaching has not been explored.

Problem-based learning (PBL) is a learner-centered pedagogy, in which students learn knowledge by solving several open-ended problems found in a trigger material, and take a center stage in case-based, self-directed learning (11,12). PBL enables students to learn more profoundly and improves students’ self-confidence and self-learning ability (13-15).

In recent years, the rapid development of AI algorithm has contributed to the innovation of pedagogy (16). Currently application of AI in teaching is mainly in the form of a robot teacher and/or a robotic examination system, and the AI system replaces teachers’ role in teaching and/or performance-evaluating. Taking the advantages of AI algorithm in image recognition and its widely use in medical field, we proposed an alternative approach of applying AI in medical teaching. We have developed an AI-tutoring PBL module, in which students learn ophthalmic knowledge through solving questions related to the CC-Cruiser, the AI pediatric cataract consultation platform developed by Zhongshan Ophthalmic Center. We validated the effectiveness of the AI-tutoring PBL module by comparing it to the traditional lecture-based teaching in ophthalmology clerkship.

Methods

Participant enrollment and assignment

Thirty-eight Grade-two students participated in ophthalmology clerkship at Sun Yat-Sen University were enrolled in August 2018. The participants were randomly assigned to two groups. In Group A, students learned congenital cataracts through an AI-tutoring problem-based-learning module by exploring and operating an AI diagnosis platform. In Group B, students learned congenital cataracts through traditional lecture given with the same faculty. All the procedures in this study were arranged strictly with the approval of the institutional review board of Zhongshan Ophthalmic Centre of Sun Yat-sen University (IRB-ZOC-SYSU). Written informed consents have been obtained from all students. Flowchart of the study was specific (Figure 1). Both groups had a pre-lecture test, a 45-minute lecture and an after-lecture test. The same examination paper was used for pre- and after-lecture test. The test consisted of two parts. Part I was about the signs and diagnosis of pediatric cataract, and Part II was about the treatment of pediatric cataract. Both groups were taught by an experienced instructor who is a physician in Home of Cataract Children in Zhongshan Ophthalmic Centre.

Figure 1 Flowchart of the study.

CC-cruiser, the AI consultation platform for pediatric cataract

CC-Cruiser is a pediatric cataract consultation platform developed by our team. This AI platform uses slit lamp photos of pediatric cataracts to identify and classify the location, density and extent of cataract lesions, and provides treatment recommendations for either conservative treatment or surgical treatment. Trained with a large number of labeled ocular images, this AI platform performed as good as ophthalmic experts in diagnosis and evaluation of pediatric cataract (7).

AI-tutoring PBL module

One day before class, students in group A got the introduction materials on CC-cruiser as well as key points for diagnosis and treatment of pediatric cataract. Besides, the tutor provided five questions on diagnosis and treatment of pediatric cataract to students and encouraged them to discuss the questions before class. In the class, 10 minutes were given to students to finish the pre-lecture test. Then the students used 25 minutes to diagnose and evaluate patient with the AI consultation platform, CC-cruiser. Finally, a 10-minute wrapped-up discussion about the diagnosis and treatment of congenital cataract was carried out.

Design of the traditional lecture

One day before class, students in group B got key points for diagnosis and treatment of pediatric cataract. The tutor provided five questions, the same as for Group A, on diagnosis and treatment of pediatric cataract to students and encouraged them to discuss the questions before class. During the class, 10 minutes were given to students to finish the pre-lecture test. Then the tutor gave a 25-minute traditional lecture on congenital cataract. Finally, a 10-minute wrapped-up discussion about the diagnosis and treatment of congenital cataract was carried out.

Student evaluations

Student evaluations of AI-tutoring PBL were measured by a 17-item questionnaire, including 16 one-choice question and one open-ended question (Table 1).

Table 1 Seventeen-item questionnaire
Full table

The questionnaire in our study were designed according to the previous studies in other subjects of medical education (17). The questionnaire was conducted in Group A after class to collect and assess student satisfaction on AI-tutoring PBL module. Information of the questionnaire have several topics, which included: knowledge acquisition (2 items), motivational dimension (3 items), group cooperation (1 item), creative and critical thinking (1 item), instructor performance (3 items), organization (1 item), overall rating (2 item), recommendations (2 items). Statements were rated on a 4-point scale ranging from “strongly agree” (with the highest score) to “strongly disagree” (with the lowest score).

Data analysis

All data were collected non-anonymously. The effects of the AI-tutoring PBL module on students’ performance were measured by comparing pre- and after-lecture scores using a paired t-test. Independent t-test was used to compare the improvement of student performance between Group A and B. A P value less than 0.050 was considered statistically significant. All analyses were conducted by SPSS software version 22.0 (SPSS Inc., Chicago, IL, USA).

Results

Baseline characteristics

All participants took part in the class and completed examinations as required. There was no difference in baseline characteristics between two groups including gender, pre-lecture scores of Part I (P=0.469), Part II (P=0.097) and total scores (P=0.158) (Table 2).

Table 2 Baseline characteristics of subjects
Full table

Improvement of student performance in each group

The post-lecture scores were significantly higher than the pre-lecture scores in both groups (Group A: P<0.001, Group B: P<0.001). The improvement of both groups in the part of sign and diagnosis test (Part I) was significantly. (Group A: P<0.001, Group B: P<0.001). However, the improvement of group A in treatment plan test (Part II) was not obvious (P=0.083) (Table 3).

Table 3 Improvement of student performance in each group
Full table

Comparison of student performance between Group A and B

The total and the Part I score of group A were higher than those of group B (P=0.034 and 0.033 respectively). Compared with the pre-lecture test, the improvement of group A in Part I was more obvious than that of group B (P=0.016), which may be the main reason why the total post-lecture score of group A was significantly higher than that of group B (Table 4).

Table 4 Comparison of student performance between Group A and B
Full table

Student satisfaction

All students in group A responded to the questionnaire. Overall, all respondents were satisfied and agreed that AI-tutoring PBL was helpful, effective, motive and beneficial to help develop critical and creative thinking. Meanwhile, they agreed that the instructor played an important role in AI-tutoring PBL curriculum (Figure 2).

Figure 2 Students’ responses to Artificial-tutoring problem-based learning courses. Each survey question used a four-point scale.

We also collected students’ answer to the open-ended question: what are the advantages and disadvantages of AI aided PBL teaching methods? Many students confirmed that AI-tutoring PBL benefits them, in terms of preview, communications, collaborations, self-learning ability and critical thinking. However, some students complained on higher requirement of hardware and that they need to spend more time previewing.

Discussion

Clerkship is a transition from general education to learning in a specific field (18). Students in this phase have the advantages of active thinking, strong learning ability, and are eager to accept new concepts and technologies (19,20). Therefore, we combined an AI diagnosis platform with a model of active learning, PBL to set up an AI-tutoring PBL module to improve student performance in ophthalmic clerkship.

We found some interesting results. Group A improved more greatly in the sign and diagnosis test (Part I) compared to Group B, which indicates that the AI model based on image features is better than traditional teaching in deepening medical students’ understanding of signs and morphological features. The AI platform does not directly replace part of the teacher’s work. What CC cruiser presents to students is the disease-evaluating ability that AI has gained through learning from human experts. The AI platform extracts the characteristics of the patient’s ocular image using convolutional neural networks for diagnosis and treatment decision-making (7). Based on the input ocular images and the corresponding evaluation made by the AI platform, the students were encouraged to discuss and collaborate to learn knowledge on diagnosis and treatment of pediatric cataract. The post-lecture scores of Part II were similar in both groups, which demonstrated that the AI module was equally effective to the traditional lecture in conveying knowledge on treatment strategy.

Our anonymous questionnaire gave a subjective assessment of AI-tutoring PBL from students in terms of knowledge acquisition, motivational dimension, cooperation, creative and critical thinking, instructor performance, and preferences. Consistent with the results of the test scores, students agreed that AI-tutoring PBL helps improve student initiative, collaboration, creativity and critical thinking. What is more, the students all agreed that the role of instructor in AI-tutoring PBL is indispensable. The instructor highly facilitated the learning process, also encouraged and provided opportunities for discussion. Consistently, studies showed that the sophisticated role of PBL tutor strikes the right balance between process facilitation and information delivery (21).

The results of this study should be assessed within the context of its limitations. Our data showed that AI-tutoring PBL module could be more effective in conveying knowledge on diseases with specific signs, such as pediatric cataract. However, the effects of this module in teaching other types of diseases require further study. Secondly, the number of participants is relatively small and the results require to be validated with larger sample size.

In summary, application of AI-tutoring PBL into ophthalmology clerkship improved students’ performance and satisfaction. AI-tutoring PBL teaching showed advantage in promoting students' understanding of signs of diseases. The instructors play an indispensable role in AI-tutoring PBL curriculum.

Acknowledgments

Funding: This study was funded by the National Key R&D Program of China (2018YFC0116500) and the National Natural Science Foundation of China (81873675).

Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editors (Haotian Lin and Limin Yu) for the series “Medical Artificial Intelligent Research” published in Annals of Translational Medicine. The article was sent for external peer review organized by the Guest Editors and the editorial office.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm.2019.12.15). The series “Medical Artificial Intelligent Research” was commissioned by the editorial office without any funding or sponsorship. HL served as the unpaid Guest Editor of the series. The other authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All the procedures in this study were arranged strictly with the approval of the institutional review board of Zhongshan Ophthalmic Centre of Sun Yat-sen University (IRB-ZOC-SYSU). Written informed consents have been obtained from all students.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Jordan MI, Mitchell TM. Machine learning: Trends, perspectives, and prospects. Science 2015;349:255-60. [Crossref] [PubMed]
2. Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60-88. [Crossref] [PubMed]
3. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017;542:115-8. [Crossref] [PubMed]
4. Takahashi H, Tampo H, Arai Y, et al. Applying artificial intelligence to disease staging: Deep learning for improved staging of diabetic retinopathy. PLoS One 2017;12:e0179790. [Crossref] [PubMed]
5. Ting DSW, Cheung CY, Lim G, et al. Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes. JAMA 2017;318:2211. [Crossref] [PubMed]
6. Gulshan V, Peng L, Coram M, et al. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA 2016;316:2402. [Crossref] [PubMed]
7. Lin H, Li R, Liu Z, et al. Diagnostic Efficacy and Therapeutic Decision-making Capacity of an Artificial Intelligence Platform for Childhood Cataracts in Eye Clinics: A Multicentre Randomized Controlled Trial. EClinicalMedicine 2019;9:52-9. [Crossref] [PubMed]
8. Wilson ML, Sayed S, Horton S, et al. Artificial intelligence can augment global pathology initiatives – Authors' reply. Lancet 2018;392:2352. [Crossref] [PubMed]
9. Ehteshami Bejnordi B, Veta M, Johannes Van Diest P, et al. Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. JAMA 2017;318:2199. [Crossref] [PubMed]
10. Golden JA. Deep Learning Algorithms for Detection of Lymph Node Metastases From Breast Cancer. JAMA 2017;318:2184. [Crossref] [PubMed]
11. Hartling L, Spooner C, Tjosvold L, et al. Problem-based learning in pre-clinical medical education: 22 years of outcome research. Med Teach 2010;32:28-35. [Crossref] [PubMed]
12. Norman GR, Schmidt HG. The psychological basis of problem-based learning: a review of the evidence. Acad Med 1992;67:557-65. [Crossref] [PubMed]
13. Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med 1993;68:550-63. [Crossref] [PubMed]
14. Faisal R, Bahadur S, Shinwari L. Problem-based learning in comparison with lecture-based learning among medical students. J Pak Med Assoc 2016;66:650-3. [PubMed]
15. Bate E, Hommes J, Duvivier R, et al. Problem-based learning (PBL): getting the most out of your students - their roles and responsibilities: AMEE Guide No. 84. Med Teach 2014;36:1-12. [Crossref] [PubMed]
16. Hassabis D, Kumaran D, Summerfield C, et al. Neuroscience-Inspired Artificial Intelligence. Neuron 2017;95:245-58. [Crossref] [PubMed]
17. Huang Z, Li M, Zhou Y, et al. Modified Team-Based Learning in an Ophthalmology Clerkship in China. PLoS One 2016;11:e0154250. [Crossref] [PubMed]
18. Adelsheimer A, Berman RS, Pachter HL, et al. Surgical clerkship or medical clerkship first: Does it make a difference? Am J Surg 2018;215:304-8. [Crossref] [PubMed]
19. Haak DC. Increased structure and active learning reduce the achievement gap in introductory biology. Science 2011;332:1213-6. [Crossref] [PubMed]
20. Freeman S, Eddy SL, McDonough M, et al. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A 2014;111:8410-5. [Crossref] [PubMed]
21. Wang Q, Li H, Pang W. From PBL tutoring to PBL coaching in undergraduate medical education: an interpretative phenomenological analysis study. Med Educ Online 2016;21:31973. [Crossref] [PubMed]