Operational neural networks
<p>Feed-forward, fully connected artificial neural networks or the so-called multi-layer perceptrons are well-known universal approximators. However, their learning performance varies significantly depending on the function or the solution space that they attempt to approximate. This is mainly...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , , |
| Published: |
2020
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513568009879552 |
|---|---|
| author | Serkan Kiranyaz (3762058) |
| author2 | Turker Ince (14150610) Alexandros Iosifidis (8967770) Moncef Gabbouj (2276533) |
| author2_role | author author author |
| author_facet | Serkan Kiranyaz (3762058) Turker Ince (14150610) Alexandros Iosifidis (8967770) Moncef Gabbouj (2276533) |
| author_role | author |
| dc.creator.none.fl_str_mv | Serkan Kiranyaz (3762058) Turker Ince (14150610) Alexandros Iosifidis (8967770) Moncef Gabbouj (2276533) |
| dc.date.none.fl_str_mv | 2020-03-06T18:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1007/s00521-020-04780-3 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Operational_neural_networks/21597069 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Artificial intelligence Software engineering Operational neural network Heterogeneous and nonlinear neural networks Convolutional neural networks |
| dc.title.none.fl_str_mv | Operational neural networks |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p>Feed-forward, fully connected artificial neural networks or the so-called multi-layer perceptrons are well-known universal approximators. However, their learning performance varies significantly depending on the function or the solution space that they attempt to approximate. This is mainly because of their homogenous configuration based solely on the linear neuron model. Therefore, while they learn very well those problems with a monotonous, relatively simple and linearly separable solution space, they may entirely fail to do so when the solution space is highly nonlinear and complex. Sharing the same linear neuron model with two additional constraints (local connections and weight sharing), this is also true for the conventional convolutional neural networks (CNNs) and it is, therefore, not surprising that in many challenging problems only the deep CNNs with a massive complexity and depth can achieve the required diversity and the learning performance. In order to address this drawback and also to accomplish a more generalized model over the convolutional neurons, this study proposes a novel network model, called operational neural networks (ONNs), which can be heterogeneous and encapsulate neurons with any set of operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. Finally, the training method to back-propagate the error through the operational layers of ONNs is formulated. Experimental results over highly challenging problems demonstrate the superior learning capabilities of ONNs even with few neurons and hidden layers. </p><h2>Other Information</h2> <p> Published in: Neural Computing and Applications<br> License: <a href="https://creativecommons.org/licenses/by/4.0" target="_blank">https://creativecommons.org/licenses/by/4.0</a><br>See article on publisher's website: <a href="http://dx.doi.org/10.1007/s00521-020-04780-3" target="_blank">http://dx.doi.org/10.1007/s00521-020-04780-3</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_fab88d0f17c519bd883c8dbbaf265146 |
| identifier_str_mv | 10.1007/s00521-020-04780-3 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/21597069 |
| publishDate | 2020 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Operational neural networksSerkan Kiranyaz (3762058)Turker Ince (14150610)Alexandros Iosifidis (8967770)Moncef Gabbouj (2276533)Information and computing sciencesArtificial intelligenceSoftware engineeringOperational neural networkHeterogeneous and nonlinear neural networksConvolutional neural networks<p>Feed-forward, fully connected artificial neural networks or the so-called multi-layer perceptrons are well-known universal approximators. However, their learning performance varies significantly depending on the function or the solution space that they attempt to approximate. This is mainly because of their homogenous configuration based solely on the linear neuron model. Therefore, while they learn very well those problems with a monotonous, relatively simple and linearly separable solution space, they may entirely fail to do so when the solution space is highly nonlinear and complex. Sharing the same linear neuron model with two additional constraints (local connections and weight sharing), this is also true for the conventional convolutional neural networks (CNNs) and it is, therefore, not surprising that in many challenging problems only the deep CNNs with a massive complexity and depth can achieve the required diversity and the learning performance. In order to address this drawback and also to accomplish a more generalized model over the convolutional neurons, this study proposes a novel network model, called operational neural networks (ONNs), which can be heterogeneous and encapsulate neurons with any set of operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. Finally, the training method to back-propagate the error through the operational layers of ONNs is formulated. Experimental results over highly challenging problems demonstrate the superior learning capabilities of ONNs even with few neurons and hidden layers. </p><h2>Other Information</h2> <p> Published in: Neural Computing and Applications<br> License: <a href="https://creativecommons.org/licenses/by/4.0" target="_blank">https://creativecommons.org/licenses/by/4.0</a><br>See article on publisher's website: <a href="http://dx.doi.org/10.1007/s00521-020-04780-3" target="_blank">http://dx.doi.org/10.1007/s00521-020-04780-3</a></p>2020-03-06T18:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1007/s00521-020-04780-3https://figshare.com/articles/journal_contribution/Operational_neural_networks/21597069CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/215970692020-03-06T18:00:00Z |
| spellingShingle | Operational neural networks Serkan Kiranyaz (3762058) Information and computing sciences Artificial intelligence Software engineering Operational neural network Heterogeneous and nonlinear neural networks Convolutional neural networks |
| status_str | publishedVersion |
| title | Operational neural networks |
| title_full | Operational neural networks |
| title_fullStr | Operational neural networks |
| title_full_unstemmed | Operational neural networks |
| title_short | Operational neural networks |
| title_sort | Operational neural networks |
| topic | Information and computing sciences Artificial intelligence Software engineering Operational neural network Heterogeneous and nonlinear neural networks Convolutional neural networks |