Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology
<h3>Objective</h3><p dir="ltr">This study aimed to compare the quality and readability of the responses generated by 3 publicly available artificial intelligence (AI) chatbots in answering frequently asked questions (FAQs) related to Oral and Maxillofacial Radiology (OMR)...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , , , |
| منشور في: |
2025
|
| الموضوعات: | |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513538608857088 |
|---|---|
| author | Dilek Helvacioglu-Yigit (15248780) |
| author2 | Husniye Demirturk (22303555) Kamran Ali (8861576) Dania Tamimi (22303558) Lisa Koenig (22303561) Abeer Almashraqi (22303564) |
| author2_role | author author author author author |
| author_facet | Dilek Helvacioglu-Yigit (15248780) Husniye Demirturk (22303555) Kamran Ali (8861576) Dania Tamimi (22303558) Lisa Koenig (22303561) Abeer Almashraqi (22303564) |
| author_role | author |
| dc.creator.none.fl_str_mv | Dilek Helvacioglu-Yigit (15248780) Husniye Demirturk (22303555) Kamran Ali (8861576) Dania Tamimi (22303558) Lisa Koenig (22303561) Abeer Almashraqi (22303564) |
| dc.date.none.fl_str_mv | 2025-04-24T12:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1016/j.oooo.2025.01.001 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Evaluating_artificial_intelligence_chatbots_for_patient_education_in_oral_and_maxillofacial_radiology/30197929 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Biomedical and clinical sciences Dentistry Health sciences Health services and systems Oral and Maxillofacial Radiology (OMR) Artificial Intelligence (AI) Chatbots Patient Education Health Communication |
| dc.title.none.fl_str_mv | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <h3>Objective</h3><p dir="ltr">This study aimed to compare the quality and readability of the responses generated by 3 publicly available artificial intelligence (AI) chatbots in answering frequently asked questions (FAQs) related to Oral and Maxillofacial Radiology (OMR) to assess their suitability for patient education. </p><h3>Study Design</h3><p dir="ltr">Fifteen OMR-related questions were selected from professional patient information websites. These questions were posed to ChatGPT-3.5 by OpenAI, Gemini 1.5 Pro by Google, and Copilot by Microsoft to generate responses. Three board-certified OMR specialists evaluated the responses regarding scientific adequacy, ease of understanding, and overall reader satisfaction. Readability was assessed using the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) scores. The Wilcoxon signed-rank test was conducted to compare the scores assigned by the evaluators to the responses from the chatbots and professional websites. Interevaluator agreement was examined by calculating the Fleiss kappa coefficient. </p><h3>Results</h3><p dir="ltr">There were no significant differences between groups in terms of scientific adequacy. In terms of readability, chatbots had overall mean FKGL and FRE scores of 12.97 and 34.11, respectively. Interevaluator agreement level was generally high. </p><h3>Conclusions</h3><p dir="ltr">Although chatbots are relatively good at responding to FAQs, validating AI-generated information using input from healthcare professionals can enhance patient care and safety. Readability of the text content in the chatbots and websites requires high reading levels.</p><h2>Other Information</h2><p dir="ltr">Published in: Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.oooo.2025.01.001" target="_blank">https://dx.doi.org/10.1016/j.oooo.2025.01.001</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_1d0bb84b8f022f9b1ea404a31327e38b |
| identifier_str_mv | 10.1016/j.oooo.2025.01.001 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/30197929 |
| publishDate | 2025 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiologyDilek Helvacioglu-Yigit (15248780)Husniye Demirturk (22303555)Kamran Ali (8861576)Dania Tamimi (22303558)Lisa Koenig (22303561)Abeer Almashraqi (22303564)Biomedical and clinical sciencesDentistryHealth sciencesHealth services and systemsOral and Maxillofacial Radiology (OMR)Artificial Intelligence (AI)ChatbotsPatient EducationHealth Communication<h3>Objective</h3><p dir="ltr">This study aimed to compare the quality and readability of the responses generated by 3 publicly available artificial intelligence (AI) chatbots in answering frequently asked questions (FAQs) related to Oral and Maxillofacial Radiology (OMR) to assess their suitability for patient education. </p><h3>Study Design</h3><p dir="ltr">Fifteen OMR-related questions were selected from professional patient information websites. These questions were posed to ChatGPT-3.5 by OpenAI, Gemini 1.5 Pro by Google, and Copilot by Microsoft to generate responses. Three board-certified OMR specialists evaluated the responses regarding scientific adequacy, ease of understanding, and overall reader satisfaction. Readability was assessed using the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) scores. The Wilcoxon signed-rank test was conducted to compare the scores assigned by the evaluators to the responses from the chatbots and professional websites. Interevaluator agreement was examined by calculating the Fleiss kappa coefficient. </p><h3>Results</h3><p dir="ltr">There were no significant differences between groups in terms of scientific adequacy. In terms of readability, chatbots had overall mean FKGL and FRE scores of 12.97 and 34.11, respectively. Interevaluator agreement level was generally high. </p><h3>Conclusions</h3><p dir="ltr">Although chatbots are relatively good at responding to FAQs, validating AI-generated information using input from healthcare professionals can enhance patient care and safety. Readability of the text content in the chatbots and websites requires high reading levels.</p><h2>Other Information</h2><p dir="ltr">Published in: Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.oooo.2025.01.001" target="_blank">https://dx.doi.org/10.1016/j.oooo.2025.01.001</a></p>2025-04-24T12:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1016/j.oooo.2025.01.001https://figshare.com/articles/journal_contribution/Evaluating_artificial_intelligence_chatbots_for_patient_education_in_oral_and_maxillofacial_radiology/30197929CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/301979292025-04-24T12:00:00Z |
| spellingShingle | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology Dilek Helvacioglu-Yigit (15248780) Biomedical and clinical sciences Dentistry Health sciences Health services and systems Oral and Maxillofacial Radiology (OMR) Artificial Intelligence (AI) Chatbots Patient Education Health Communication |
| status_str | publishedVersion |
| title | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_full | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_fullStr | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_full_unstemmed | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_short | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| title_sort | Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology |
| topic | Biomedical and clinical sciences Dentistry Health sciences Health services and systems Oral and Maxillofacial Radiology (OMR) Artificial Intelligence (AI) Chatbots Patient Education Health Communication |