A Multi-Channel Convolutional Neural Network approach to automate the citation screening process
<p dir="ltr">The systematic literature review (SLR) process is separated into several steps to increase rigor and reproducibility. The selection of primary studies (i.e., citation screening) is an important step in the SLR process. The citation screening process aims to identify the...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , |
| Published: |
2021
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513547795431424 |
|---|---|
| author | Raymon van Dinter (10521952) |
| author2 | Cagatay Catal (6897842) Bedir Tekinerdogan (6897839) |
| author2_role | author author |
| author_facet | Raymon van Dinter (10521952) Cagatay Catal (6897842) Bedir Tekinerdogan (6897839) |
| author_role | author |
| dc.creator.none.fl_str_mv | Raymon van Dinter (10521952) Cagatay Catal (6897842) Bedir Tekinerdogan (6897839) |
| dc.date.none.fl_str_mv | 2021-11-01T00:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1016/j.asoc.2021.107765 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/A_Multi-Channel_Convolutional_Neural_Network_approach_to_automate_the_citation_screening_process/24420484 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Computer vision and multimedia computation Machine learning Language, communication and culture Linguistics Systematic literature review (SLR) Citation screening Automation Neural networks Natural language processing |
| dc.title.none.fl_str_mv | A Multi-Channel Convolutional Neural Network approach to automate the citation screening process |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">The systematic literature review (SLR) process is separated into several steps to increase rigor and reproducibility. The selection of primary studies (i.e., citation screening) is an important step in the SLR process. The citation screening process aims to identify the relevant primary studies fairly and with high rigor using selection criteria. Through the study selection criteria, reviewers determine whether an article should be included or excluded from the SLR. However, the screening process is highly time-consuming and error-prone as the researchers must read each title and possibly hundreds to thousands of abstracts and full-text documents. This study aims to automate the citation screening process using Deep Learning algorithms. With this, it is aimed to reduce the time and costs of the citation screening process and increase the precision and recall of the relevant primary studies. A Multi-Channel Convolutional Neural Network (CNN) is proposed, which can automatically classify a given set of citations. As the architecture uses the title and abstract as features, our end-to-end pipeline is domain-independent. We have performed six experiments to assess the performance of Multi-Channel CNNs across 20 publicly available systematic literature review datasets. It was shown that for 18 out of 20 review datasets, the proposed method achieved significant workload savings of at least 10%, while in several cases, our model yielded a statistically significantly better performance over two benchmark review datasets. We conclude that Multi-Channel CNNs are effective for the citation screening process in SLRs. Multi-Channel CNNs perform best on large datasets of over 2500 samples with few abstracts missing.</p><h2>Other Information</h2><p dir="ltr">Published in: Applied Soft Computing<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.asoc.2021.107765" target="_blank">https://dx.doi.org/10.1016/j.asoc.2021.107765</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_85efd09d12da001a53807fd946ff8cd5 |
| identifier_str_mv | 10.1016/j.asoc.2021.107765 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/24420484 |
| publishDate | 2021 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | A Multi-Channel Convolutional Neural Network approach to automate the citation screening processRaymon van Dinter (10521952)Cagatay Catal (6897842)Bedir Tekinerdogan (6897839)Information and computing sciencesComputer vision and multimedia computationMachine learningLanguage, communication and cultureLinguisticsSystematic literature review (SLR)Citation screeningAutomationNeural networksNatural language processing<p dir="ltr">The systematic literature review (SLR) process is separated into several steps to increase rigor and reproducibility. The selection of primary studies (i.e., citation screening) is an important step in the SLR process. The citation screening process aims to identify the relevant primary studies fairly and with high rigor using selection criteria. Through the study selection criteria, reviewers determine whether an article should be included or excluded from the SLR. However, the screening process is highly time-consuming and error-prone as the researchers must read each title and possibly hundreds to thousands of abstracts and full-text documents. This study aims to automate the citation screening process using Deep Learning algorithms. With this, it is aimed to reduce the time and costs of the citation screening process and increase the precision and recall of the relevant primary studies. A Multi-Channel Convolutional Neural Network (CNN) is proposed, which can automatically classify a given set of citations. As the architecture uses the title and abstract as features, our end-to-end pipeline is domain-independent. We have performed six experiments to assess the performance of Multi-Channel CNNs across 20 publicly available systematic literature review datasets. It was shown that for 18 out of 20 review datasets, the proposed method achieved significant workload savings of at least 10%, while in several cases, our model yielded a statistically significantly better performance over two benchmark review datasets. We conclude that Multi-Channel CNNs are effective for the citation screening process in SLRs. Multi-Channel CNNs perform best on large datasets of over 2500 samples with few abstracts missing.</p><h2>Other Information</h2><p dir="ltr">Published in: Applied Soft Computing<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.asoc.2021.107765" target="_blank">https://dx.doi.org/10.1016/j.asoc.2021.107765</a></p>2021-11-01T00:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1016/j.asoc.2021.107765https://figshare.com/articles/journal_contribution/A_Multi-Channel_Convolutional_Neural_Network_approach_to_automate_the_citation_screening_process/24420484CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/244204842021-11-01T00:00:00Z |
| spellingShingle | A Multi-Channel Convolutional Neural Network approach to automate the citation screening process Raymon van Dinter (10521952) Information and computing sciences Computer vision and multimedia computation Machine learning Language, communication and culture Linguistics Systematic literature review (SLR) Citation screening Automation Neural networks Natural language processing |
| status_str | publishedVersion |
| title | A Multi-Channel Convolutional Neural Network approach to automate the citation screening process |
| title_full | A Multi-Channel Convolutional Neural Network approach to automate the citation screening process |
| title_fullStr | A Multi-Channel Convolutional Neural Network approach to automate the citation screening process |
| title_full_unstemmed | A Multi-Channel Convolutional Neural Network approach to automate the citation screening process |
| title_short | A Multi-Channel Convolutional Neural Network approach to automate the citation screening process |
| title_sort | A Multi-Channel Convolutional Neural Network approach to automate the citation screening process |
| topic | Information and computing sciences Computer vision and multimedia computation Machine learning Language, communication and culture Linguistics Systematic literature review (SLR) Citation screening Automation Neural networks Natural language processing |