Presented by: Matthew Steidle MD
University of Rochester Urology

Introduction

Recent advances in artificial intelligence have led to the creation of chatbots, such as ChatGPT.  Chatbots function as alternatives to search engines by providing detailed and articulate responses in easy to understand, conversational prose, to make information easier to digest.  However, there are rising concerns that AI-powered chatbots can sometimes present invented information that is false.  These are called “hallucinations.”  We predict that, similarly to searching for information on the internet, as chatbots improve and become increasingly prevalent, patients will turn to them to provide information on how to prevent kidney stones. We tested several of the currently publicly available chatbots to test if they provide accurate dietary advice to patients for kidney stone prevention.

Materials

The AUA guidelines on medical management of kidney stones include statements 8-11 which are labeled diet therapies.  Several publicly available chatbots were tested. The question: “how do you prevent kidney stones by changing your diet?” was entered.  The responses were then evaluated to see if they covered guidelines statements 8-11.  The responses were subjectively evaluated for accuracy of content, quality of sources, ease of understanding, and functionality. The chatbots were given an opportunity to provide further detail with the prompt: “can you elaborate on that further?”

Results

,

With only 1 exception, all of the chatbots gave dietary advice that covered the AUA dietary guidelines for kidney stone prevention.  Some gave more specifics and quantifiable targets.  For example, You.com Chat recommended drinking enough fluid to produce 2-2.5L of urine while Chat GPT gave a list of specific oxalate rich foods to avoid. See table for further details.

Conclusion

We demonstrated that by and large, AI-powered chatbots provide helpful, general information about kidney stone prevention, including hydration, a healthy balanced diet, weight loss, and avoidance of animal protein. In the present study, we found the chatbots to be easy to use and a good starting point for patient education at minimal cost.  They all recommended speaking with a dietary professional or urologist for further information.  This is important as nuances of kidney stone prevention relies heavily on stone analysis, blood work and 24-hour urinalysis. 

Funding

None

Lead Authors

Rajat Jain, MD
URMC Urology

Co-Authors

Steven Hassig, MD
URMC Urology

Scott Quarrier, MD
URMC Urology