Performance better for easy versus difficult questions
By Lori Solomon HealthDay Reporter
THURSDAY, Aug. 29, 2024 (HealthDay News) — ChatGPT provides acceptable responses to nearly half of patient urology questions, according to a study published in the Sept. 1 issue of Urology Practice.
Michael Scott, M.D., from the Stanford University School of Medicine in Palo Alto, California, and colleagues evaluated the quality of ChatGPT responses to real-world urology patient messages. The analysis included 100 electronic patient messages, with ChatGPT answers independently evaluated by five urologists.
The researchers found that 47 percent of responses were determined to be acceptable to send to patients. ChatGPT performance was better on easy versus difficult questions (56 versus 34 percent acceptable). Compared with difficult questions, responses to easy questions were more accurate, complete, helpful, and intelligible.
“ChatGPT generated acceptable responses to nearly 50 percent of patient messages with better performance for easy questions compared to difficult questions. Use of ChatGPT to help respond to patient messages can help to decrease the time burden for the care team and improve wellness,” the authors write. “Artificial intelligence performance will likely continue to improve with advances in generative artificial intelligence technology.”
Editorial (subscription or payment may be required)
Copyright © 2024 HealthDay. All rights reserved.
