Extreme Early Image Recognition Using Event-Based Vision
<p dir="ltr">While deep learning algorithms have advanced to a great extent, they are all designed for frame-based imagers that capture images at a high frame rate, which leads to a high storage requirement, heavy computations, and very high power consumption. Unlike frame-based imag...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , |
| منشور في: |
2023
|
| الموضوعات: | |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513509239291904 |
|---|---|
| author | Abubakar Abubakar (18278998) |
| author2 | AlKhzami AlHarami (18278995) Yin Yang (35103) Amine Bermak (1895947) |
| author2_role | author author author |
| author_facet | Abubakar Abubakar (18278998) AlKhzami AlHarami (18278995) Yin Yang (35103) Amine Bermak (1895947) |
| author_role | author |
| dc.creator.none.fl_str_mv | Abubakar Abubakar (18278998) AlKhzami AlHarami (18278995) Yin Yang (35103) Amine Bermak (1895947) |
| dc.date.none.fl_str_mv | 2023-07-06T03:00:00Z |
| dc.identifier.none.fl_str_mv | 10.3390/s23136195 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Extreme_Early_Image_Recognition_Using_Event-Based_Vision/26510233 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Computer vision and multimedia computation Machine learning convolutional neural network early image recognition event-based camera sensors |
| dc.title.none.fl_str_mv | Extreme Early Image Recognition Using Event-Based Vision |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">While deep learning algorithms have advanced to a great extent, they are all designed for frame-based imagers that capture images at a high frame rate, which leads to a high storage requirement, heavy computations, and very high power consumption. Unlike frame-based imagers, event-based imagers output asynchronous pixel events without the need for global exposure time, therefore lowering both power consumption and latency. In this paper, we propose an innovative image recognition technique that operates on image events rather than frame-based data, paving the way for a new paradigm of recognizing objects prior to image acquisition. To the best of our knowledge, this is the first time such a concept is introduced featuring not only extreme early image recognition but also reduced computational overhead, storage requirement, and power consumption. Our collected event-based dataset using CeleX imager and five public event-based datasets are used to prove this concept, and the testing metrics reflect how early the neural network (NN) detects an image before the full-frame image is captured. It is demonstrated that, on average for all the datasets, the proposed technique recognizes an image 38.7 ms before the first perfect event and 603.4 ms before the last event is received, which is a reduction of 34% and 69% of the time needed, respectively. Further, less processing is required as the image is recognized 9460 events earlier, which is 37% less than waiting for the first perfectly recognized image. An enhanced NN method is also introduced to reduce this time.</p><h2>Other Information</h2><p dir="ltr">Published in: Sensors<br>License: <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank">https://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.3390/s23136195" target="_blank">https://dx.doi.org/10.3390/s23136195</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_4382fc26883a57b38a1da98724412349 |
| identifier_str_mv | 10.3390/s23136195 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/26510233 |
| publishDate | 2023 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Extreme Early Image Recognition Using Event-Based VisionAbubakar Abubakar (18278998)AlKhzami AlHarami (18278995)Yin Yang (35103)Amine Bermak (1895947)Information and computing sciencesComputer vision and multimedia computationMachine learningconvolutional neural networkearly image recognitionevent-based camerasensors<p dir="ltr">While deep learning algorithms have advanced to a great extent, they are all designed for frame-based imagers that capture images at a high frame rate, which leads to a high storage requirement, heavy computations, and very high power consumption. Unlike frame-based imagers, event-based imagers output asynchronous pixel events without the need for global exposure time, therefore lowering both power consumption and latency. In this paper, we propose an innovative image recognition technique that operates on image events rather than frame-based data, paving the way for a new paradigm of recognizing objects prior to image acquisition. To the best of our knowledge, this is the first time such a concept is introduced featuring not only extreme early image recognition but also reduced computational overhead, storage requirement, and power consumption. Our collected event-based dataset using CeleX imager and five public event-based datasets are used to prove this concept, and the testing metrics reflect how early the neural network (NN) detects an image before the full-frame image is captured. It is demonstrated that, on average for all the datasets, the proposed technique recognizes an image 38.7 ms before the first perfect event and 603.4 ms before the last event is received, which is a reduction of 34% and 69% of the time needed, respectively. Further, less processing is required as the image is recognized 9460 events earlier, which is 37% less than waiting for the first perfectly recognized image. An enhanced NN method is also introduced to reduce this time.</p><h2>Other Information</h2><p dir="ltr">Published in: Sensors<br>License: <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank">https://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.3390/s23136195" target="_blank">https://dx.doi.org/10.3390/s23136195</a></p>2023-07-06T03:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.3390/s23136195https://figshare.com/articles/journal_contribution/Extreme_Early_Image_Recognition_Using_Event-Based_Vision/26510233CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/265102332023-07-06T03:00:00Z |
| spellingShingle | Extreme Early Image Recognition Using Event-Based Vision Abubakar Abubakar (18278998) Information and computing sciences Computer vision and multimedia computation Machine learning convolutional neural network early image recognition event-based camera sensors |
| status_str | publishedVersion |
| title | Extreme Early Image Recognition Using Event-Based Vision |
| title_full | Extreme Early Image Recognition Using Event-Based Vision |
| title_fullStr | Extreme Early Image Recognition Using Event-Based Vision |
| title_full_unstemmed | Extreme Early Image Recognition Using Event-Based Vision |
| title_short | Extreme Early Image Recognition Using Event-Based Vision |
| title_sort | Extreme Early Image Recognition Using Event-Based Vision |
| topic | Information and computing sciences Computer vision and multimedia computation Machine learning convolutional neural network early image recognition event-based camera sensors |