Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology
ObjectiveThis study aimed to compare the quality and readability of the responses generated by 3 publicly available artificial intelligence (AI) chatbots in answering frequently asked questions (FAQs) related to Oral and Maxillofacial Radiology (OMR) to assess their suitability for patient education...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , , , |
| التنسيق: | article |
| منشور في: |
2025
|
| الموضوعات: | |
| الوصول للمادة أونلاين: | http://dx.doi.org/10.1016/j.oooo.2025.01.001 https://www.sciencedirect.com/science/article/pii/S2212440325000033 http://hdl.handle.net/10576/64060 |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1857415083633672192 |
|---|---|
| author | Helvacioglu-Yigit, Dilek |
| author2 | Demirturk, Husniye Ali, Kamran Tamimi, Dania Koenig, Lisa Almashraqi, Abeer |
| author2_role | author author author author author |
| author_facet | Helvacioglu-Yigit, Dilek Demirturk, Husniye Ali, Kamran Tamimi, Dania Koenig, Lisa Almashraqi, Abeer |
| author_role | author |
| dc.creator.none.fl_str_mv | Helvacioglu-Yigit, Dilek Demirturk, Husniye Ali, Kamran Tamimi, Dania Koenig, Lisa Almashraqi, Abeer |
| dc.date.none.fl_str_mv | 2025-04-08T07:22:26Z 2025-01-11 |
| dc.format.none.fl_str_mv | application/pdf |
| dc.identifier.none.fl_str_mv | http://dx.doi.org/10.1016/j.oooo.2025.01.001 2212-4403 https://www.sciencedirect.com/science/article/pii/S2212440325000033 http://hdl.handle.net/10576/64060 750-759 2212-4411 |
| dc.language.none.fl_str_mv | en |
| dc.publisher.none.fl_str_mv | Elsevier |
| dc.rights.none.fl_str_mv | http://creativecommons.org/licenses/by/4.0/ info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Artificial Intelligence (AI) Chatbots Oral and Maxillofacial Radiology (OMR) Readability (FKGL & FRE Scores) Patient Education Scientific Adequacy |
| dc.title.none.fl_str_mv | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| dc.type.none.fl_str_mv | Article info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/article |
| description | ObjectiveThis study aimed to compare the quality and readability of the responses generated by 3 publicly available artificial intelligence (AI) chatbots in answering frequently asked questions (FAQs) related to Oral and Maxillofacial Radiology (OMR) to assess their suitability for patient education. Study DesignFifteen OMR-related questions were selected from professional patient information websites. These questions were posed to ChatGPT-3.5 by OpenAI, Gemini 1.5 Pro by Google, and Copilot by Microsoft to generate responses. Three board-certified OMR specialists evaluated the responses regarding scientific adequacy, ease of understanding, and overall reader satisfaction. Readability was assessed using the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) scores. The Wilcoxon signed-rank test was conducted to compare the scores assigned by the evaluators to the responses from the chatbots and professional websites. Interevaluator agreement was examined by calculating the Fleiss kappa coefficient. ResultsThere were no significant differences between groups in terms of scientific adequacy. In terms of readability, chatbots had overall mean FKGL and FRE scores of 12.97 and 34.11, respectively. Interevaluator agreement level was generally high. ConclusionsAlthough chatbots are relatively good at responding to FAQs, validating AI-generated information using input from healthcare professionals can enhance patient care and safety. Readability of the text content in the chatbots and websites requires high reading levels. |
| eu_rights_str_mv | openAccess |
| format | article |
| id | qu_fe8d3daa2a4637560bd526ad11d91d43 |
| identifier_str_mv | 2212-4403 750-759 2212-4411 |
| language_invalid_str_mv | en |
| network_acronym_str | qu |
| network_name_str | Qatar University repository |
| oai_identifier_str | oai:qspace.qu.edu.qa:10576/64060 |
| publishDate | 2025 |
| publisher.none.fl_str_mv | Elsevier |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | http://creativecommons.org/licenses/by/4.0/ |
| spelling | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiologyHelvacioglu-Yigit, DilekDemirturk, HusniyeAli, KamranTamimi, DaniaKoenig, LisaAlmashraqi, AbeerArtificial Intelligence (AI) ChatbotsOral and Maxillofacial Radiology (OMR)Readability (FKGL & FRE Scores)Patient EducationScientific AdequacyObjectiveThis study aimed to compare the quality and readability of the responses generated by 3 publicly available artificial intelligence (AI) chatbots in answering frequently asked questions (FAQs) related to Oral and Maxillofacial Radiology (OMR) to assess their suitability for patient education. Study DesignFifteen OMR-related questions were selected from professional patient information websites. These questions were posed to ChatGPT-3.5 by OpenAI, Gemini 1.5 Pro by Google, and Copilot by Microsoft to generate responses. Three board-certified OMR specialists evaluated the responses regarding scientific adequacy, ease of understanding, and overall reader satisfaction. Readability was assessed using the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) scores. The Wilcoxon signed-rank test was conducted to compare the scores assigned by the evaluators to the responses from the chatbots and professional websites. Interevaluator agreement was examined by calculating the Fleiss kappa coefficient. ResultsThere were no significant differences between groups in terms of scientific adequacy. In terms of readability, chatbots had overall mean FKGL and FRE scores of 12.97 and 34.11, respectively. Interevaluator agreement level was generally high. ConclusionsAlthough chatbots are relatively good at responding to FAQs, validating AI-generated information using input from healthcare professionals can enhance patient care and safety. Readability of the text content in the chatbots and websites requires high reading levels.Open Access funding provided by the Qatar National Library.Elsevier2025-04-08T07:22:26Z2025-01-11Articleinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttp://dx.doi.org/10.1016/j.oooo.2025.01.0012212-4403https://www.sciencedirect.com/science/article/pii/S2212440325000033http://hdl.handle.net/10576/64060750-7592212-4411enhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:qspace.qu.edu.qa:10576/640602025-06-16T19:05:37Z |
| spellingShingle | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology Helvacioglu-Yigit, Dilek Artificial Intelligence (AI) Chatbots Oral and Maxillofacial Radiology (OMR) Readability (FKGL & FRE Scores) Patient Education Scientific Adequacy |
| status_str | publishedVersion |
| title | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_full | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_fullStr | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_full_unstemmed | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_short | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_sort | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| topic | Artificial Intelligence (AI) Chatbots Oral and Maxillofacial Radiology (OMR) Readability (FKGL & FRE Scores) Patient Education Scientific Adequacy |
| url | http://dx.doi.org/10.1016/j.oooo.2025.01.001 https://www.sciencedirect.com/science/article/pii/S2212440325000033 http://hdl.handle.net/10576/64060 |