Super Neurons
<p dir="ltr">Self-Organized Operational Neural Networks (Self-ONNs) have recently been proposed as new-generation neural network models with nonlinear learning units, i.e., the generative neurons that yield an elegant level of diversity; however, like its predecessor, conventional Co...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , , , , , |
| منشور في: |
2023
|
| الموضوعات: | |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513527015800832 |
|---|---|
| author | Serkan Kiranyaz (3762058) |
| author2 | Junaid Malik (16869930) Mehmet Yamac (17986993) Mert Duman (17986996) Ilke Adalioglu (17986999) Esin Guldogan (17987002) Turker Ince (14150610) Moncef Gabbouj (2276533) |
| author2_role | author author author author author author author |
| author_facet | Serkan Kiranyaz (3762058) Junaid Malik (16869930) Mehmet Yamac (17986993) Mert Duman (17986996) Ilke Adalioglu (17986999) Esin Guldogan (17987002) Turker Ince (14150610) Moncef Gabbouj (2276533) |
| author_role | author |
| dc.creator.none.fl_str_mv | Serkan Kiranyaz (3762058) Junaid Malik (16869930) Mehmet Yamac (17986993) Mert Duman (17986996) Ilke Adalioglu (17986999) Esin Guldogan (17987002) Turker Ince (14150610) Moncef Gabbouj (2276533) |
| dc.date.none.fl_str_mv | 2023-10-09T12:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1109/tetci.2023.3314658 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Super_Neurons/25243378 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Artificial intelligence Mathematical sciences Numerical and computational mathematics Kernel Neurons Training Biological neural networks Computational intelligence Biological system modeling Location awareness Convolutional neural networks generative neurons non-localized kernels operational neural networks receptive field |
| dc.title.none.fl_str_mv | Super Neurons |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">Self-Organized Operational Neural Networks (Self-ONNs) have recently been proposed as new-generation neural network models with nonlinear learning units, i.e., the generative neurons that yield an elegant level of diversity; however, like its predecessor, conventional Convolutional Neural Networks (CNNs), they still have a common drawback: localized (fixed) kernel operations. This severely limits the receptive field and information flow between layers and thus brings the necessity for deep and complex models. It is highly desired to improve the receptive field size without increasing the kernel dimensions. This requires a significant upgrade over the generative neurons to achieve the “non-localized kernel operations” for each connection between consecutive layers. In this article, we present superior (generative) neuron models (or super neurons in short) that allow random or learnable kernel shifts and thus can increase the receptive field size of each connection. The kernel localization process varies among the two super-neuron models. The first model assumes randomly localized kernels within a range and the second one learns (optimizes) the kernel locations during training. An extensive set of comparative evaluations against conventional and deformable convolutional, along with the generative neurons demonstrates that super neurons can empower Self-ONNs to achieve a superior learning and generalization capability with a minimal computational complexity burden. PyTorch implementation of Self-ONNs with super-neurons is now publically shared.</p><h2>Other Information</h2><p dir="ltr">Published in: IEEE Transactions on Emerging Topics in Computational Intelligence<br>License: <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank">https://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1109/tetci.2023.3314658" target="_blank">https://dx.doi.org/10.1109/tetci.2023.3314658</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_84292a0601d12f9b3de4f2e5650f4139 |
| identifier_str_mv | 10.1109/tetci.2023.3314658 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/25243378 |
| publishDate | 2023 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Super NeuronsSerkan Kiranyaz (3762058)Junaid Malik (16869930)Mehmet Yamac (17986993)Mert Duman (17986996)Ilke Adalioglu (17986999)Esin Guldogan (17987002)Turker Ince (14150610)Moncef Gabbouj (2276533)Information and computing sciencesArtificial intelligenceMathematical sciencesNumerical and computational mathematicsKernelNeuronsTrainingBiological neural networksComputational intelligenceBiological system modelingLocation awarenessConvolutional neural networksgenerative neuronsnon-localized kernelsoperational neural networksreceptive field<p dir="ltr">Self-Organized Operational Neural Networks (Self-ONNs) have recently been proposed as new-generation neural network models with nonlinear learning units, i.e., the generative neurons that yield an elegant level of diversity; however, like its predecessor, conventional Convolutional Neural Networks (CNNs), they still have a common drawback: localized (fixed) kernel operations. This severely limits the receptive field and information flow between layers and thus brings the necessity for deep and complex models. It is highly desired to improve the receptive field size without increasing the kernel dimensions. This requires a significant upgrade over the generative neurons to achieve the “non-localized kernel operations” for each connection between consecutive layers. In this article, we present superior (generative) neuron models (or super neurons in short) that allow random or learnable kernel shifts and thus can increase the receptive field size of each connection. The kernel localization process varies among the two super-neuron models. The first model assumes randomly localized kernels within a range and the second one learns (optimizes) the kernel locations during training. An extensive set of comparative evaluations against conventional and deformable convolutional, along with the generative neurons demonstrates that super neurons can empower Self-ONNs to achieve a superior learning and generalization capability with a minimal computational complexity burden. PyTorch implementation of Self-ONNs with super-neurons is now publically shared.</p><h2>Other Information</h2><p dir="ltr">Published in: IEEE Transactions on Emerging Topics in Computational Intelligence<br>License: <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank">https://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1109/tetci.2023.3314658" target="_blank">https://dx.doi.org/10.1109/tetci.2023.3314658</a></p>2023-10-09T12:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1109/tetci.2023.3314658https://figshare.com/articles/journal_contribution/Super_Neurons/25243378CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/252433782023-10-09T12:00:00Z |
| spellingShingle | Super Neurons Serkan Kiranyaz (3762058) Information and computing sciences Artificial intelligence Mathematical sciences Numerical and computational mathematics Kernel Neurons Training Biological neural networks Computational intelligence Biological system modeling Location awareness Convolutional neural networks generative neurons non-localized kernels operational neural networks receptive field |
| status_str | publishedVersion |
| title | Super Neurons |
| title_full | Super Neurons |
| title_fullStr | Super Neurons |
| title_full_unstemmed | Super Neurons |
| title_short | Super Neurons |
| title_sort | Super Neurons |
| topic | Information and computing sciences Artificial intelligence Mathematical sciences Numerical and computational mathematics Kernel Neurons Training Biological neural networks Computational intelligence Biological system modeling Location awareness Convolutional neural networks generative neurons non-localized kernels operational neural networks receptive field |