Presented by: Rafael Gevorkyan MSc
University of Southern California, Keck School of Medicine

Introduction

The complex language used in clinical reports, such as prostate biopsy results, often poses a challenge to patient comprehension. As artificial intelligence (AI) continues to evolve, its potential for facilitating patient understanding has been explored. This study set out to assess the quality of patient-centered reports generated by ChatGPT, with evaluations provided by urological surgeons. 

Materials

25 original prostate biopsy reports, 5 from each Gleason Grade group, were transformed into patient-centered versions using ChatGPT-3.5. The generated reports were evaluated by 7 urological surgeons using a 7-item questionnaire that focused on structure, clarity of diagnosis, accuracy of biopsy result description, extent of disease explanation, Gleason grade interpretation, comparative effectiveness for patient understanding, and willingness to use AI-generated reports in practice. Ratings were scored on a 5-point Likert scale. We utilized 2 readability measures, the Flesch Reading Ease and the Flesch-Kincaid Grade-Level, to evaluate and compare the readability of each ChatGPT-generated report against its respective original pathology report. 

Results

,

The ChatGPT-generated patient-centered reports exhibited significantly higher Flesch Reading Ease Scores (49.05 ± 4.75) compared to the original biopsy reports (31.15 ± 14.62; p < 0.001). Flesch-Kincaid Grade Levels were similar between the two (p=0.940). Surgeons’ evaluations of the ChatGPT-generated reports indicated favorable responses across the seven-item criteria. Surgeons strongly agreed with the generated reports’ use of patient-friendly language (48.6%), accuracy of biopsy results (54.9%), disease extent explanation (56.6%), and Gleason grade interpretation (58.9%). Surgeons indicated a neutral to positive agreement that the ChatGPT reports’ structure and potential to improve patients’ understanding of their disease, yet a strong agreement for willingness to use them in their practice (76.0%).  

Conclusion

The evaluation of ChatGPT-generated patient-centered reports demonstrated improved readability compared to the original biopsy reports. As the field of AI continues to advance, tools like ChatGPT may play a key role in patient-centered care. 

Funding

No funding.

Co-Authors

Rafael Gevorkyan, MS
University of California, Irvine, Department of Urology

Narmina Khanmammadova, MD
University of California, Irvine, Department of Urology

Maria Epino, BS
University of California, Irvine, Department of Urology

Tuan Thanh Nguyen, MD, MSc
University of Medicine and Pharmacy at Ho Chi Minh City

Sohrab Naushad Ali, MD, MSc, FRCSC
University of California, Irvine, Department of Urology

Andrei D Cumpanas, MD
University of California, Irvine, Department of Urology

Jacob Basilius, MD
University of California, Irvine, Department of Urology

Antonio Gorgen, MD
University of California, Irvine, Department of Urology

Jaime Altamirano, MD
University of California, Irvine, Department of Urology

Mohammed Shahait, MBBS
Clemenceau Medical Center, Department of Surgery

Thomas Ahlering, MD
University of California, Irvine, Department of Urology

David I Lee, MD, FACS
University of California, Irvine, Department of Urology