Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches
<p dir="ltr">Delphi studies in disaster medicine lack consensus on expert agreement metrics. This study examined various metrics using a Delphi study on chemical, biological, radiological, and nuclear (CBRN) preparedness in the Middle East and North Africa region. Forty international...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , , , , , , , , , , , , , |
| Published: |
2025
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513534660968448 |
|---|---|
| author | Hassan Farhat (9000509) |
| author2 | Alan M. Batt (15373346) Mariana Helou (12477762) Heejun Shin (3069867) James Laughton (14778532) Carolyn Dumbeck (22466161) Arezoo Dehghani (22466164) Fatemeh Rezaei (2742127) Nidaa Bajow (12522916) Luc Mortelmans (2387137) Walid Abougalala (17300914) Roberto Mugavero (22466167) Gregory Ciottone (8480562) Guillaume Alinier (6952004) Mohamed Ben Dhiab (21632843) |
| author2_role | author author author author author author author author author author author author author author |
| author_facet | Hassan Farhat (9000509) Alan M. Batt (15373346) Mariana Helou (12477762) Heejun Shin (3069867) James Laughton (14778532) Carolyn Dumbeck (22466161) Arezoo Dehghani (22466164) Fatemeh Rezaei (2742127) Nidaa Bajow (12522916) Luc Mortelmans (2387137) Walid Abougalala (17300914) Roberto Mugavero (22466167) Gregory Ciottone (8480562) Guillaume Alinier (6952004) Mohamed Ben Dhiab (21632843) |
| author_role | author |
| dc.creator.none.fl_str_mv | Hassan Farhat (9000509) Alan M. Batt (15373346) Mariana Helou (12477762) Heejun Shin (3069867) James Laughton (14778532) Carolyn Dumbeck (22466161) Arezoo Dehghani (22466164) Fatemeh Rezaei (2742127) Nidaa Bajow (12522916) Luc Mortelmans (2387137) Walid Abougalala (17300914) Roberto Mugavero (22466167) Gregory Ciottone (8480562) Guillaume Alinier (6952004) Mohamed Ben Dhiab (21632843) |
| dc.date.none.fl_str_mv | 2025-04-06T03:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1111/1468-5973.70044 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Evaluating_the_Performance_of_Agreement_Metrics_in_a_Delphi_Study_on_Chemical_Biological_Radiological_and_Nuclear_Major_Incidents_Preparedness_Using_Classical_and_Machine_Learning_Approaches/30405271 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Health sciences Health services and systems agreement analysis Delphi study disaster medicine expert's opinion MENA |
| dc.title.none.fl_str_mv | Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">Delphi studies in disaster medicine lack consensus on expert agreement metrics. This study examined various metrics using a Delphi study on chemical, biological, radiological, and nuclear (CBRN) preparedness in the Middle East and North Africa region. Forty international disaster medicine experts evaluated 133 items across ten CBRN Preparedness Assessment Tool themes using a 5‐point Likert scale. Agreement was measured using Kendall's W, Intraclass Correlation Coefficient, and Cohen's Kappa. Statistical and machine learning techniques compared metric performance. The overall agreement mean score was 4.91 ± 0.71, with 89.21% average agreement. Kappa emerged as the most sensitive metric in statistical and machine learning analyses, with a feature importance score of 168.32. The Kappa coefficient showed variations across CBRN PAT themes, including medical protocols, logistics, and infrastructure. The integrated statistical and machine learning approach provides a promising method for understanding expert consensus in disaster preparedness, with potential for future refinement by incorporating additional contextual factors.</p><h2>Other Information</h2><p dir="ltr">Published in: Journal of Contingencies and Crisis Management<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1111/1468-5973.70044" target="_blank">https://dx.doi.org/10.1111/1468-5973.70044</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_7bc1b2882b452711f9859835ea8fea78 |
| identifier_str_mv | 10.1111/1468-5973.70044 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/30405271 |
| publishDate | 2025 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning ApproachesHassan Farhat (9000509)Alan M. Batt (15373346)Mariana Helou (12477762)Heejun Shin (3069867)James Laughton (14778532)Carolyn Dumbeck (22466161)Arezoo Dehghani (22466164)Fatemeh Rezaei (2742127)Nidaa Bajow (12522916)Luc Mortelmans (2387137)Walid Abougalala (17300914)Roberto Mugavero (22466167)Gregory Ciottone (8480562)Guillaume Alinier (6952004)Mohamed Ben Dhiab (21632843)Health sciencesHealth services and systemsagreement analysisDelphi studydisaster medicineexpert's opinionMENA<p dir="ltr">Delphi studies in disaster medicine lack consensus on expert agreement metrics. This study examined various metrics using a Delphi study on chemical, biological, radiological, and nuclear (CBRN) preparedness in the Middle East and North Africa region. Forty international disaster medicine experts evaluated 133 items across ten CBRN Preparedness Assessment Tool themes using a 5‐point Likert scale. Agreement was measured using Kendall's W, Intraclass Correlation Coefficient, and Cohen's Kappa. Statistical and machine learning techniques compared metric performance. The overall agreement mean score was 4.91 ± 0.71, with 89.21% average agreement. Kappa emerged as the most sensitive metric in statistical and machine learning analyses, with a feature importance score of 168.32. The Kappa coefficient showed variations across CBRN PAT themes, including medical protocols, logistics, and infrastructure. The integrated statistical and machine learning approach provides a promising method for understanding expert consensus in disaster preparedness, with potential for future refinement by incorporating additional contextual factors.</p><h2>Other Information</h2><p dir="ltr">Published in: Journal of Contingencies and Crisis Management<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1111/1468-5973.70044" target="_blank">https://dx.doi.org/10.1111/1468-5973.70044</a></p>2025-04-06T03:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1111/1468-5973.70044https://figshare.com/articles/journal_contribution/Evaluating_the_Performance_of_Agreement_Metrics_in_a_Delphi_Study_on_Chemical_Biological_Radiological_and_Nuclear_Major_Incidents_Preparedness_Using_Classical_and_Machine_Learning_Approaches/30405271CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/304052712025-04-06T03:00:00Z |
| spellingShingle | Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches Hassan Farhat (9000509) Health sciences Health services and systems agreement analysis Delphi study disaster medicine expert's opinion MENA |
| status_str | publishedVersion |
| title | Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches |
| title_full | Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches |
| title_fullStr | Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches |
| title_full_unstemmed | Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches |
| title_short | Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches |
| title_sort | Evaluating the Performance of Agreement Metrics in a Delphi Study on Chemical, Biological, Radiological and Nuclear Major Incidents Preparedness Using Classical and Machine Learning Approaches |
| topic | Health sciences Health services and systems agreement analysis Delphi study disaster medicine expert's opinion MENA |