Presented by: Rafael Gevorkyan MSc
University of Southern California, Keck School of Medicine

Introduction

The potential of Artificial Intelligence (AI) to translate complex medical reports into patient-friendly language has been of interest. This study aims to assess the clinical utility of ChatGPT in generating patient-centered surgical pathology reports for patients who underwent radical prostatectomy. 

Materials

The study encompassed 25 original surgical pathology reports evenly distributed across each Gleason Grade group. The reports were converted into patient-centered versions using ChatGPT-3.5. These generated reports were reviewed by 7 urological surgeons. An 8-item questionnaire was designed to evaluate the quality of these reports, with each item scored on a 5-point Likert scale. The items assessed the structure, readability, accuracy of pathology results, disease extent explanation, Gleason grade interpretation, surgical margin explanation, patient understanding improvement, and the surgeons' willingness to use these AI reports in their clinical practice. The Flesch Reading Ease score and the Flesch-Kincaid Grade-Level score were computed and compared between the ChatGPT-generated and original reports to assess the relative readability and comprehension level. 

Results

,

The Flesch Reading Ease score improved in the ChatGPT-generated reports (49.64 ± 3.39) compared to the original reports (41.39 ± 16.82; p=0.020). However, the Flesch-Kincaid Grade Level was higher in the ChatGPT-generated reports (10.47 ± 0.70) than the original reports (8.72 ± 2.17; p<0.001). Surgeons’ evaluations of the ChatGPT-generated reports indicated favorable responses across the eight-item criteria. Surgeons strongly agreed with the generated reports' structure (71.4%), readability (80.0%), accuracy of pathology results (78.9%), and disease extent explanation (53.7%). Neutral to positive responses were observed for surgical margin explanation and Gleason grade interpretation. A strong agreement existed for the ChatGPT-generated reports' potential to improve patient understanding (63.4%) and surgeons' willingness to use them in their practice (87.4%). 

Conclusion

Our results suggest that ChatGPT-generated reports may enhance patient comprehension and have the potential to be utilized in clinical settings. There is strong willingness among surgeons to use these reports in their practice.  

Funding

No funding,

Co-Authors

Rafael Gevorkyan, MS
University of California, Irvine, Department of Urology

Mohammed Shahait, MD
Clemenceau Medical Center, Department of Surgery

Maria Epino, BS
University of California, Irvine, Department of Urology

Narmina Khanmammadova, MD
University of California, Irvine, Department of Urology

Tuan Thanh Nguyen, MD, MSc
University of Medicine and Pharmacy at Ho Chi Minh City

Sohrab Naushad Ali, MD, MSc, FRCSC
University of California, Irvine, Department of Urology

Jacob Basilius, MD
University of California, Irvine, Department of Urology

Jaime Altamirano, MD
University of California, Irvine, Department of Urology

Antonio Gorgen, MD
University of California, Irvine, Department of Urology

Andrei D. Cumpanas, MD
University of California, Irvine, Department of Urology

Thomas Ahlering, MD
University of California, Irvine, Department of Urology

David I Lee, MD, FACS
University of California, Irvine, Department of Urology