<< Back to the abstract archive
Automated Quantitative Assessment of Letters of Recommendation for Integrated Plastic Surgery Residency: Linguistic Shifts in the COVID-era
Tripp Leavitt, Richard Lee, Rachel Ng, Steven Bernard
Cleveland Clinic
2022-01-15
Presenter: Tripp Leavitt
Affidavit:
I certify that the material proposed for presentation in this abstract has not been published in any scientific journal or previously presented at a major meeting. The program director is responsible for making a statement within the confines of the box below specific to how much of the work on this project represents the original work of the resident. All authors/submitters of each abstract should discuss this with their respective program director for accurate submission of information as well as the program director's approval for inclusion of his/her electronic signature.
Director Name: Steven Bernard
Author Category: Resident Plastic Surgery
Presentation Category: Clinical
Abstract Category: General Reconstruction
Numerous studies have demonstrated the importance of letters of recommendation as part of the residency application, particularly among surgical subspecialties, including plastic surgery. However, objective assessment remains difficult for a variety of reasons, not least the sheer volume of applications and letters in a given year. Adoption of the standard ACAPS form letter is variable and the narrative component is usually where the qualifications of a stellar applicant truly come to light. In order to provide a more quantitative assessment of the longer narrative component, we constructed a code-enabled pipeline combining optical character recognition (OCR), applicant anonymization, and linguistic software. This allowed automated processing of the letters of recommendation through a custom dictionary previously developed within our institution, containing 400 terms across 24 categories. As a proof-of-concept, letters submitted to our integrated residency program from pre- and post-COVID application cycles were compared. Across the four application cycles from 2018 to 2021, a total of 2492 letters were analyzed (1201 pre- and 1291 post-COVID eras, respectively). Average word counts were approximately equal across both groups, (mean word count 790.2 vs 798.8). Terms categorized as grindstone, social, agentic orientation, and agentic personality were more prevalent in the post-COVID era group (p<0.05). Standout terms such as "best, leader, and prodigy" were no more or less likely to be used across the groups. Future applications of natural language processing may further contextualize current findings.
