Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts?
<h3 dir="ltr">Introduction</h3><p dir="ltr">Chat Generative Pre-Trained Transformer (ChatGPT) is a natural language processing model. It can be argued that ChatGPT has recently begun to assume the role of a technological assistant capable of supporting academics...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , , , , , |
| Published: |
2025
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513525511094272 |
|---|---|
| author | Ahmet Öztürk (19903851) |
| author2 | Anılcan Tahsin Karahan (22928833) Serkan Günay (21781960) Abdul Samed Erdal (22928836) Seval Komut (22928839) Erdal Komut (22928842) Yavuz Yiğit (17823383) |
| author2_role | author author author author author author |
| author_facet | Ahmet Öztürk (19903851) Anılcan Tahsin Karahan (22928833) Serkan Günay (21781960) Abdul Samed Erdal (22928836) Seval Komut (22928839) Erdal Komut (22928842) Yavuz Yiğit (17823383) |
| author_role | author |
| dc.creator.none.fl_str_mv | Ahmet Öztürk (19903851) Anılcan Tahsin Karahan (22928833) Serkan Günay (21781960) Abdul Samed Erdal (22928836) Seval Komut (22928839) Erdal Komut (22928842) Yavuz Yiğit (17823383) |
| dc.date.none.fl_str_mv | 2025-08-07T09:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1016/j.ajem.2025.07.034 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Artificial_intelligence_as_author_Can_scientific_reviewers_recognize_GPT-4o-generated_manuscripts_/30971599 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Artificial intelligence Philosophy and religious studies Applied ethics ChatGPT Artificial intelligence Journal Editor Reviewer |
| dc.title.none.fl_str_mv | Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts? |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <h3 dir="ltr">Introduction</h3><p dir="ltr">Chat Generative Pre-Trained Transformer (ChatGPT) is a natural language processing model. It can be argued that ChatGPT has recently begun to assume the role of a technological assistant capable of supporting academics in the process of scientific writing. ChatGPT may contribute to the spread of incorrect or incomplete information within academic literature, leading to conceptual confusion and potential academic misconduct. The aim of this study is to determine whether a scientific article entirely generated by an AI application such as ChatGPT can be detected by an academic journal editor or peer reviewer.</p><h3 dir="ltr">Methods</h3><p dir="ltr">This study was conducted between November 1, 2024, and December 1, 2024. GPT-4o, was utilized in this study. ChatGPT was instructed to write a scientific article focused on predicting mortality and return of spontaneous circulation (ROSC) in OHCA cases. The manuscript written by ChatGPT-4o was sent to 14 different reviewers who had previously served as reviewers or editors. The reviewers were asked to evaluate the manuscript as if they were an SCI-E journal editor or peer reviewer. The reviewers were informed that the article had been written by ChatGPT and were asked whether they had identified this during their review.</p><h3 dir="ltr">Results</h3><p dir="ltr">Among the reviewers, 42.9 % (<i>n</i> = 6) decided to reject the manuscript at the editorial stage, whereas another 42.9 % (<i>n</i>= 6) opted to forward it to a peer reviewer. During the peer review stage, 42.9 % (<i>n</i> = 6) of the reviewers recommended rejection, while 28.6 % (<i>n</i> = 4) suggested major revisions. 78.6 % (<i>n</i> = 11) of the reviewers did not realize that the manuscript had been generated by an artificial intelligence model.</p><h3 dir="ltr">Conclusion</h3><p dir="ltr">The findings of our study highlight the necessity for journal editors and peer reviewers to be well-informed about ChatGPT and to develop systems capable of identifying whether a manuscript has been written by a human or generated by artificial intelligence.</p><h2 dir="ltr">Other Information</h2><p dir="ltr">Published in: The American Journal of Emergency Medicine<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.ajem.2025.07.034" target="_blank">https://dx.doi.org/10.1016/j.ajem.2025.07.034</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_47b6ba5c294cdff953d0ad8b0ebd7521 |
| identifier_str_mv | 10.1016/j.ajem.2025.07.034 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/30971599 |
| publishDate | 2025 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts?Ahmet Öztürk (19903851)Anılcan Tahsin Karahan (22928833)Serkan Günay (21781960)Abdul Samed Erdal (22928836)Seval Komut (22928839)Erdal Komut (22928842)Yavuz Yiğit (17823383)Information and computing sciencesArtificial intelligencePhilosophy and religious studiesApplied ethicsChatGPTArtificial intelligenceJournalEditorReviewer<h3 dir="ltr">Introduction</h3><p dir="ltr">Chat Generative Pre-Trained Transformer (ChatGPT) is a natural language processing model. It can be argued that ChatGPT has recently begun to assume the role of a technological assistant capable of supporting academics in the process of scientific writing. ChatGPT may contribute to the spread of incorrect or incomplete information within academic literature, leading to conceptual confusion and potential academic misconduct. The aim of this study is to determine whether a scientific article entirely generated by an AI application such as ChatGPT can be detected by an academic journal editor or peer reviewer.</p><h3 dir="ltr">Methods</h3><p dir="ltr">This study was conducted between November 1, 2024, and December 1, 2024. GPT-4o, was utilized in this study. ChatGPT was instructed to write a scientific article focused on predicting mortality and return of spontaneous circulation (ROSC) in OHCA cases. The manuscript written by ChatGPT-4o was sent to 14 different reviewers who had previously served as reviewers or editors. The reviewers were asked to evaluate the manuscript as if they were an SCI-E journal editor or peer reviewer. The reviewers were informed that the article had been written by ChatGPT and were asked whether they had identified this during their review.</p><h3 dir="ltr">Results</h3><p dir="ltr">Among the reviewers, 42.9 % (<i>n</i> = 6) decided to reject the manuscript at the editorial stage, whereas another 42.9 % (<i>n</i>= 6) opted to forward it to a peer reviewer. During the peer review stage, 42.9 % (<i>n</i> = 6) of the reviewers recommended rejection, while 28.6 % (<i>n</i> = 4) suggested major revisions. 78.6 % (<i>n</i> = 11) of the reviewers did not realize that the manuscript had been generated by an artificial intelligence model.</p><h3 dir="ltr">Conclusion</h3><p dir="ltr">The findings of our study highlight the necessity for journal editors and peer reviewers to be well-informed about ChatGPT and to develop systems capable of identifying whether a manuscript has been written by a human or generated by artificial intelligence.</p><h2 dir="ltr">Other Information</h2><p dir="ltr">Published in: The American Journal of Emergency Medicine<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.ajem.2025.07.034" target="_blank">https://dx.doi.org/10.1016/j.ajem.2025.07.034</a></p>2025-08-07T09:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1016/j.ajem.2025.07.034https://figshare.com/articles/journal_contribution/Artificial_intelligence_as_author_Can_scientific_reviewers_recognize_GPT-4o-generated_manuscripts_/30971599CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/309715992025-08-07T09:00:00Z |
| spellingShingle | Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts? Ahmet Öztürk (19903851) Information and computing sciences Artificial intelligence Philosophy and religious studies Applied ethics ChatGPT Artificial intelligence Journal Editor Reviewer |
| status_str | publishedVersion |
| title | Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts? |
| title_full | Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts? |
| title_fullStr | Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts? |
| title_full_unstemmed | Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts? |
| title_short | Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts? |
| title_sort | Artificial intelligence as author: Can scientific reviewers recognize GPT-4o-generated manuscripts? |
| topic | Information and computing sciences Artificial intelligence Philosophy and religious studies Applied ethics ChatGPT Artificial intelligence Journal Editor Reviewer |