<< Back to the abstract archive
Evaluating Healthcare-Based AI for PSITE and Its Potential Educational Impact in Plastic Surgery Training
Mykal Gerald, MD, Anusha Singh, BS, Jaeda Robinson, BS, John Mastronardo, BBA, Cristiane Ueno, MD
The Ohio State Wexner Medical Center
2025-01-04
Presenter: Mykal J Gerald, MD
Affidavit:
This project represents the original work of Mykal Gerald, MD. She contributed 75% of the project idea and > 60% on the project itself.
Director Name: Cristiane Ueno
Author Category: Resident Plastic Surgery
Presentation Category: Clinical
Abstract Category: General Reconstruction
Background: In recent literature, artificial intelligence (AI), has proven to be a positive adjunctive tool in healthcare and research. In an effort to enhance plastic surgery in-service training exam (PSITE) preparation, this study aims to uncover the AI platform that most accurately answers PSITE questions. As AI continues to be developed, new healthcare specific platforms are entering the industry, creating an opportunity to advance the healthcare system as a whole.
Methods: To conduct this study, the most common AI platforms, ChatGPT 4.0 and Copilot, were used in addition to Open Evidence AI, which is a healthcare specific AI platform. A total of 54 questions from the 2023 PSITE questions were input into the AI platforms thus far. Questions that depended on image interpretation were excluded from the study. Open Evidence AI did not recognize multiple choice questions and thus required reconfiguration of the question to obtain an answer. To confirm accuracy, answers obtained from each AI platform were compared to the verified answers per the PSITE answer key.
Results: Text-based PSITE question accuracy varied between platforms: ChatGPT 4.0 (83.33%), Open Evidence AI (75.93%), and Copilot (68.52%).
Conclusion: ChatGPT 4.0 demonstrated the highest accuracy, suggesting its value as a study tool for plastic surgery residents. Limitations of this study include modifying multiple choice questions for Open Evidence AI and inability to test image-based questions. As new healthcare-based AI platforms emerge and AI platform use increases, its accuracy should improve, thus continued evaluation of their efficacy is necessary.
