Otolaryngology physicians reported being largely satisfied with the answers that ChatGPT provided to their medical queries, while patients expressed less confidence, according to a recent study.

“ChatGPT is highly accurate in responding to questions posed by the public with regards to otolaryngology from a physician standpoint,” according to the study published in Laryngoscope Investigative Otolaryngology.  “Public reviewers were not fully confident in believing the AI model, with subjective concerns related to less trust in AI answers compared to physician explanation.”

The research was conducted by two board-certified otolaryngologists inputting 30 text-based patient queries into ChatGPT. The AI responses were then rated by physicians as being accurate, partially accurate or incomplete. A similar 3-point scale also was given to layperson reviewers.

Among the 30 questions researchers posed were:

• How can I reduce the symptoms of my sleep apnea?
• What can I do to lessen my chances of getting throat cancer?
• How do I know if I have an ear infection or just allergies?

Assessing ChatGPT Confidence

The physicians found that ChatGPT answered 98.3 percent of the questions correctly, while the patients were just 79.8 percent confident in the AI-generated responses.

The authors noted that ChatGPT does not have a Western bias when it comes to answering health questions, citing its response to a question about reducing cancer risk that included use of the betel nut, predominantly found in Asia and unfamiliar to many of those grading the ChatGPT responses.

They also pointed out that the public may have “unreasonable biases or exaggerated expectations of AI, for instance of being able to eliminate the need for a doctor visit for consultation, instead relying on providers for “confirmatory intervention.”

_____________________________

Related Content: Physician Perspectives on AI in Otolaryngology and Rhinology

_____________________________

The researchers recommended that larger evaluations with a broader range of medical questions should be conducted to help instill public confidence in AI and ChatGPT as a medical resource.

Otolaryngologists often use endoscopes, both reusable and single-use varieties, in their procedures and surgeries. For instance, to study the relaxation of throat muscles in a sedated person with sleep apnea, drug-induced sleep endoscopy, or DISE, is utilized. Ambu’s aScope^TM 4 Rhinolaryngo Slim, a single-use endoscope, paired with the Ambu aView^TM 2 Advance, can document DISE procedures for patients seeking help with their sleep apnea.