A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures
<p dir="ltr">In recent years, there has been a significant advancement in memristor-based neural networks, positioning them as a pivotal processing-in-memory deployment architecture for a wide array of deep learning applications. Within this realm of progress, the emerging parallel a...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , , , , |
| منشور في: |
2024
|
| الموضوعات: | |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513509702762496 |
|---|---|
| author | Bo Lyu (16522643) |
| author2 | Yin Yang (35103) Yuting Cao (4231810) Tuo Shi (3137919) Yiran Chen (4787181) Tingwen Huang (7168691) Shiping Wen (7168688) |
| author2_role | author author author author author author |
| author_facet | Bo Lyu (16522643) Yin Yang (35103) Yuting Cao (4231810) Tuo Shi (3137919) Yiran Chen (4787181) Tingwen Huang (7168691) Shiping Wen (7168688) |
| author_role | author |
| dc.creator.none.fl_str_mv | Bo Lyu (16522643) Yin Yang (35103) Yuting Cao (4231810) Tuo Shi (3137919) Yiran Chen (4787181) Tingwen Huang (7168691) Shiping Wen (7168688) |
| dc.date.none.fl_str_mv | 2024-04-19T12:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1016/j.neunet.2024.106312 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/A_memristive_all-inclusive_hypernetwork_for_parallel_analog_deployment_of_full_search_space_architectures/26491063 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Distributed computing and systems software Machine learning Memristor Convolutional neural network Neural architecture search Hypernetwork |
| dc.title.none.fl_str_mv | A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">In recent years, there has been a significant advancement in memristor-based neural networks, positioning them as a pivotal processing-in-memory deployment architecture for a wide array of deep learning applications. Within this realm of progress, the emerging parallel analog memristive platforms are prominent for their ability to generate multiple feature maps in a single processing cycle. However, a notable limitation is that they are specifically tailored for neural networks with fixed structures. As an orthogonal direction, recent research reveals that neural architecture should be specialized for tasks and deployment platforms. Building upon this, the neural architecture search (NAS) methods effectively explore promising architectures in a large design space. However, these NAS-based architectures are generally heterogeneous and diversified, making it challenging for deployment on current single-prototype, customized, parallel analog memristive hardware circuits. Therefore, investigating memristive analog deployment that overrides the full search space is a promising and challenging problem. Inspired by this, and beginning with the <i>DARTS</i> search space, we study the memristive hardware design of primitive operations and propose the memristive all-inclusive hypernetwork that covers 2 × 1 0 <sup>25</sup> network architectures. Our computational simulation results on 3 representative architectures (<i>DARTS-V1, DARTS-V2, PDARTS</i>) show that our memristive all-inclusive hypernetwork achieves promising results on the CIFAR10 dataset (89.2 % of <i>PDARTS</i> with 8-bit quantization precision), and is compatible with all architectures in the <i>DARTS </i>full-space. The hardware performance simulation indicates that the memristive all-inclusive hypernetwork costs slightly more resource consumption (nearly the same in power, 22 % ∼ 25 % increase in Latency, 1 . 5 × in Area) relative to the individual deployment, which is reasonable and may reach a tolerable trade-off deployment scheme for industrial scenarios.</p><h2>Other Information</h2><p dir="ltr">Published in: Neural Networks<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.neunet.2024.106312" target="_blank">https://dx.doi.org/10.1016/j.neunet.2024.106312</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_7488442e34c07e4351ba993d3320cbbb |
| identifier_str_mv | 10.1016/j.neunet.2024.106312 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/26491063 |
| publishDate | 2024 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architecturesBo Lyu (16522643)Yin Yang (35103)Yuting Cao (4231810)Tuo Shi (3137919)Yiran Chen (4787181)Tingwen Huang (7168691)Shiping Wen (7168688)Information and computing sciencesDistributed computing and systems softwareMachine learningMemristorConvolutional neural networkNeural architecture searchHypernetwork<p dir="ltr">In recent years, there has been a significant advancement in memristor-based neural networks, positioning them as a pivotal processing-in-memory deployment architecture for a wide array of deep learning applications. Within this realm of progress, the emerging parallel analog memristive platforms are prominent for their ability to generate multiple feature maps in a single processing cycle. However, a notable limitation is that they are specifically tailored for neural networks with fixed structures. As an orthogonal direction, recent research reveals that neural architecture should be specialized for tasks and deployment platforms. Building upon this, the neural architecture search (NAS) methods effectively explore promising architectures in a large design space. However, these NAS-based architectures are generally heterogeneous and diversified, making it challenging for deployment on current single-prototype, customized, parallel analog memristive hardware circuits. Therefore, investigating memristive analog deployment that overrides the full search space is a promising and challenging problem. Inspired by this, and beginning with the <i>DARTS</i> search space, we study the memristive hardware design of primitive operations and propose the memristive all-inclusive hypernetwork that covers 2 × 1 0 <sup>25</sup> network architectures. Our computational simulation results on 3 representative architectures (<i>DARTS-V1, DARTS-V2, PDARTS</i>) show that our memristive all-inclusive hypernetwork achieves promising results on the CIFAR10 dataset (89.2 % of <i>PDARTS</i> with 8-bit quantization precision), and is compatible with all architectures in the <i>DARTS </i>full-space. The hardware performance simulation indicates that the memristive all-inclusive hypernetwork costs slightly more resource consumption (nearly the same in power, 22 % ∼ 25 % increase in Latency, 1 . 5 × in Area) relative to the individual deployment, which is reasonable and may reach a tolerable trade-off deployment scheme for industrial scenarios.</p><h2>Other Information</h2><p dir="ltr">Published in: Neural Networks<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1016/j.neunet.2024.106312" target="_blank">https://dx.doi.org/10.1016/j.neunet.2024.106312</a></p>2024-04-19T12:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1016/j.neunet.2024.106312https://figshare.com/articles/journal_contribution/A_memristive_all-inclusive_hypernetwork_for_parallel_analog_deployment_of_full_search_space_architectures/26491063CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/264910632024-04-19T12:00:00Z |
| spellingShingle | A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures Bo Lyu (16522643) Information and computing sciences Distributed computing and systems software Machine learning Memristor Convolutional neural network Neural architecture search Hypernetwork |
| status_str | publishedVersion |
| title | A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures |
| title_full | A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures |
| title_fullStr | A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures |
| title_full_unstemmed | A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures |
| title_short | A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures |
| title_sort | A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures |
| topic | Information and computing sciences Distributed computing and systems software Machine learning Memristor Convolutional neural network Neural architecture search Hypernetwork |