Ensemble Deep Random Vector Functional Link Neural Network for Regression
<p dir="ltr">Inspired by the ensemble strategy of machine learning, deep random vector functional link (dRVFL), and ensemble dRVFL (edRVFL) has shown state-of-the-art results on different datasets. Our present work first fills the gap of dRVFL and edRVFL work in the field of regressi...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , , |
| Published: |
2022
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513560959254528 |
|---|---|
| author | Minghui Hu (2457952) |
| author2 | Jet Herng Chion (16904667) Ponnuthurai Nagaratnam Suganthan (11274636) Rakesh Kumar Katuwal (16904670) |
| author2_role | author author author |
| author_facet | Minghui Hu (2457952) Jet Herng Chion (16904667) Ponnuthurai Nagaratnam Suganthan (11274636) Rakesh Kumar Katuwal (16904670) |
| author_role | author |
| dc.creator.none.fl_str_mv | Minghui Hu (2457952) Jet Herng Chion (16904667) Ponnuthurai Nagaratnam Suganthan (11274636) Rakesh Kumar Katuwal (16904670) |
| dc.date.none.fl_str_mv | 2022-11-10T00:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1109/tsmc.2022.3213628 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Ensemble_Deep_Random_Vector_Functional_Link_Neural_Network_for_Regression/24056343 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Artificial intelligence Machine learning Neurons Training Biological neural networks Reservoirs Stochastic processes Mathematical models Task analysis Ensemble Neural networks random vector functional link (RVFL) Regression |
| dc.title.none.fl_str_mv | Ensemble Deep Random Vector Functional Link Neural Network for Regression |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">Inspired by the ensemble strategy of machine learning, deep random vector functional link (dRVFL), and ensemble dRVFL (edRVFL) has shown state-of-the-art results on different datasets. Our present work first fills the gap of dRVFL and edRVFL work in the field of regression. We test and evaluate the performances of the dRVFLs on regression problems. Subsequently, we propose a novel regularization method [boosted factor (BF)], two dRVFLs variants [edRVFL with skip connection (edRVFL-SC) and edRVFL with random skip connections (edRVFL-RSC)] and one strategy [ensemble skip connection edRVFL (esc-edRVFL)] which show significant improvement over the original dRVFL. The BF is a newly introduced hyperparameter to scale the values of the activated hidden neurons to accommodate the diversity of the data, and it is also able to filter the neurons. edRVFL-SC and edRVFL-RSC are the edRVFL variants with skip connections. In edRVFL-SC, we apply dense skip connections to the edRVFL, which is inspired by the residual architecture in the deep learning area. However, due to the specificity of randomized networks, the simple skip connections are probably leading to the reuse of useless features. To address this problem, we propose a random skip connection-based edRVFL, which can keep the diversity in the latent space. esc-RVFL is an ensemble scheme that utilizes several edRVFL-RSC models trained on the different folds of the training dataset. The esc-edRVFL is identified as the best-performing algorithm through a comprehensive evaluation of 31 UCI datasets.</p><h2>Other Information</h2><p dir="ltr">Published in: IEEE Transactions on Systems, Man, and Cybernetics: Systems<br>License: <a href="https://creativecommons.org/licenses/by/4.0/legalcode" target="_blank">https://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1109/tsmc.2022.3213628" target="_blank">https://dx.doi.org/10.1109/tsmc.2022.3213628</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_6e8b17bf0037a567912a4abae6c6095f |
| identifier_str_mv | 10.1109/tsmc.2022.3213628 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/24056343 |
| publishDate | 2022 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Ensemble Deep Random Vector Functional Link Neural Network for RegressionMinghui Hu (2457952)Jet Herng Chion (16904667)Ponnuthurai Nagaratnam Suganthan (11274636)Rakesh Kumar Katuwal (16904670)Information and computing sciencesArtificial intelligenceMachine learningNeuronsTrainingBiological neural networksReservoirsStochastic processesMathematical modelsTask analysisEnsembleNeural networksrandom vector functional link (RVFL)Regression<p dir="ltr">Inspired by the ensemble strategy of machine learning, deep random vector functional link (dRVFL), and ensemble dRVFL (edRVFL) has shown state-of-the-art results on different datasets. Our present work first fills the gap of dRVFL and edRVFL work in the field of regression. We test and evaluate the performances of the dRVFLs on regression problems. Subsequently, we propose a novel regularization method [boosted factor (BF)], two dRVFLs variants [edRVFL with skip connection (edRVFL-SC) and edRVFL with random skip connections (edRVFL-RSC)] and one strategy [ensemble skip connection edRVFL (esc-edRVFL)] which show significant improvement over the original dRVFL. The BF is a newly introduced hyperparameter to scale the values of the activated hidden neurons to accommodate the diversity of the data, and it is also able to filter the neurons. edRVFL-SC and edRVFL-RSC are the edRVFL variants with skip connections. In edRVFL-SC, we apply dense skip connections to the edRVFL, which is inspired by the residual architecture in the deep learning area. However, due to the specificity of randomized networks, the simple skip connections are probably leading to the reuse of useless features. To address this problem, we propose a random skip connection-based edRVFL, which can keep the diversity in the latent space. esc-RVFL is an ensemble scheme that utilizes several edRVFL-RSC models trained on the different folds of the training dataset. The esc-edRVFL is identified as the best-performing algorithm through a comprehensive evaluation of 31 UCI datasets.</p><h2>Other Information</h2><p dir="ltr">Published in: IEEE Transactions on Systems, Man, and Cybernetics: Systems<br>License: <a href="https://creativecommons.org/licenses/by/4.0/legalcode" target="_blank">https://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1109/tsmc.2022.3213628" target="_blank">https://dx.doi.org/10.1109/tsmc.2022.3213628</a></p>2022-11-10T00:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1109/tsmc.2022.3213628https://figshare.com/articles/journal_contribution/Ensemble_Deep_Random_Vector_Functional_Link_Neural_Network_for_Regression/24056343CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/240563432022-11-10T00:00:00Z |
| spellingShingle | Ensemble Deep Random Vector Functional Link Neural Network for Regression Minghui Hu (2457952) Information and computing sciences Artificial intelligence Machine learning Neurons Training Biological neural networks Reservoirs Stochastic processes Mathematical models Task analysis Ensemble Neural networks random vector functional link (RVFL) Regression |
| status_str | publishedVersion |
| title | Ensemble Deep Random Vector Functional Link Neural Network for Regression |
| title_full | Ensemble Deep Random Vector Functional Link Neural Network for Regression |
| title_fullStr | Ensemble Deep Random Vector Functional Link Neural Network for Regression |
| title_full_unstemmed | Ensemble Deep Random Vector Functional Link Neural Network for Regression |
| title_short | Ensemble Deep Random Vector Functional Link Neural Network for Regression |
| title_sort | Ensemble Deep Random Vector Functional Link Neural Network for Regression |
| topic | Information and computing sciences Artificial intelligence Machine learning Neurons Training Biological neural networks Reservoirs Stochastic processes Mathematical models Task analysis Ensemble Neural networks random vector functional link (RVFL) Regression |