An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge
Camera traps deployed in remote locations provide an effective method for ecologists to monitor and study wildlife in a non-invasive way. However, current camera traps suffer from two problems. First, the images are manually classified and counted, which is expensive. Second, due to manual coding, t...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , , , |
| التنسيق: | article |
| منشور في: |
2022
|
| الموضوعات: | |
| الوصول للمادة أونلاين: | https://hdl.handle.net/11073/33268 |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513431859625984 |
|---|---|
| author | Zaulkernan, Imran |
| author2 | Dhou, Salam Judas, Jacky Sajun, Ali Reza Gomez, Brylle Ryan Hussain, Lana Alhaj |
| author2_role | author author author author author |
| author_facet | Zaulkernan, Imran Dhou, Salam Judas, Jacky Sajun, Ali Reza Gomez, Brylle Ryan Hussain, Lana Alhaj |
| author_role | author |
| dc.creator.none.fl_str_mv | Zaulkernan, Imran Dhou, Salam Judas, Jacky Sajun, Ali Reza Gomez, Brylle Ryan Hussain, Lana Alhaj |
| dc.date.none.fl_str_mv | 2022-01-13 2026-03-26T08:58:07Z 2026-03-26T08:58:07Z |
| dc.format.none.fl_str_mv | application/pdf |
| dc.identifier.none.fl_str_mv | Zualkernan, I., Dhou, S., Judas, J., Sajun, A. R., Gomez, B. R., & Hussain, L. A. (2022). An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge. Computers, 11(1), 13. https://doi.org/10.3390/computers11010013 2073-431X https://hdl.handle.net/11073/33268 10.3390/computers11010013 |
| dc.language.none.fl_str_mv | en |
| dc.publisher.none.fl_str_mv | MDPI |
| dc.relation.none.fl_str_mv | https://doi.org/10.3390/computers11010013 |
| dc.rights.none.fl_str_mv | Attribution 4.0 International http://creativecommons.org/licenses/by/4.0/ |
| dc.subject.none.fl_str_mv | Deep learning Animal classification Image classification Internet of Things Image processing Edge computing Animal surveillance Wildlife monitoring Edge computing |
| dc.title.none.fl_str_mv | An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge |
| dc.type.none.fl_str_mv | Peer-Reviewed Published version info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/article |
| description | Camera traps deployed in remote locations provide an effective method for ecologists to monitor and study wildlife in a non-invasive way. However, current camera traps suffer from two problems. First, the images are manually classified and counted, which is expensive. Second, due to manual coding, the results are often stale by the time they get to the ecologists. Using the Internet of Things (IoT) combined with deep learning represents a good solution for both these problems, as the images can be classified automatically, and the results immediately made available to ecologists. This paper proposes an IoT architecture that uses deep learning on edge devices to convey animal classification results to a mobile app using the LoRaWAN low-power, wide-area network. The primary goal of the proposed approach is to reduce the cost of the wildlife monitoring process for ecologists, and to provide real-time animal sightings data from the camera traps in the field. Camera trap image data consisting of 66,400 images were used to train the InceptionV3, MobileNetV2, ResNet18, EfficientNetB1, DenseNet121, and Xception neural network models. While performance of the trained models was statistically different (Kruskal–Wallis: Accuracy H(5) = 22.34, p < 0.05; F1-score H(5) = 13.82, p = 0.0168), there was only a 3% difference in the F1-score between the worst (MobileNet V2) and the best model (Xception). Moreover, the models made similar errors (Adjusted Rand Index (ARI) > 0.88 and Adjusted Mutual Information (AMU) > 0.82). Subsequently, the best model, Xception (Accuracy = 96.1%; F1-score = 0.87; F1-Score = 0.97 with oversampling), was optimized and deployed on the Raspberry Pi, Google Coral, and Nvidia Jetson edge devices using both TenorFlow Lite and TensorRT frameworks. Optimizing the models to run on edge devices reduced the average macro F1-Score to 0.7, and adversely affected the minority classes, reducing their F1-score to as low as 0.18. Upon stress testing, by processing 1000 images consecutively, Jetson Nano, running a TensorRT model, outperformed others with a latency of 0.276 s/image (s.d. = 0.002) while consuming an average current of 1665.21 mA. Raspberry Pi consumed the least average current (838.99 mA) with a ten times worse latency of 2.83 s/image (s.d. = 0.036). Nano was the only reasonable option as an edge device because it could capture most animals whose maximum speeds were below 80 km/h, including goats, lions, ostriches, etc. While the proposed architecture is viable, unbalanced data remain a challenge and the results can potentially be improved by using object detection to reduce imbalances and by exploring semi-supervised learning. |
| format | article |
| id | aus_24424c82a006d9311e2c3c680b9eb495 |
| identifier_str_mv | Zualkernan, I., Dhou, S., Judas, J., Sajun, A. R., Gomez, B. R., & Hussain, L. A. (2022). An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge. Computers, 11(1), 13. https://doi.org/10.3390/computers11010013 2073-431X 10.3390/computers11010013 |
| language_invalid_str_mv | en |
| network_acronym_str | aus |
| network_name_str | aus |
| oai_identifier_str | oai:repository.aus.edu:11073/33268 |
| publishDate | 2022 |
| publisher.none.fl_str_mv | MDPI |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | Attribution 4.0 International http://creativecommons.org/licenses/by/4.0/ |
| spelling | An IoT System Using Deep Learning to Classify Camera Trap Images on the EdgeZaulkernan, ImranDhou, SalamJudas, JackySajun, Ali RezaGomez, Brylle RyanHussain, Lana AlhajDeep learningAnimal classificationImage classificationInternet of ThingsImage processingEdge computingAnimal surveillanceWildlife monitoringEdge computingCamera traps deployed in remote locations provide an effective method for ecologists to monitor and study wildlife in a non-invasive way. However, current camera traps suffer from two problems. First, the images are manually classified and counted, which is expensive. Second, due to manual coding, the results are often stale by the time they get to the ecologists. Using the Internet of Things (IoT) combined with deep learning represents a good solution for both these problems, as the images can be classified automatically, and the results immediately made available to ecologists. This paper proposes an IoT architecture that uses deep learning on edge devices to convey animal classification results to a mobile app using the LoRaWAN low-power, wide-area network. The primary goal of the proposed approach is to reduce the cost of the wildlife monitoring process for ecologists, and to provide real-time animal sightings data from the camera traps in the field. Camera trap image data consisting of 66,400 images were used to train the InceptionV3, MobileNetV2, ResNet18, EfficientNetB1, DenseNet121, and Xception neural network models. While performance of the trained models was statistically different (Kruskal–Wallis: Accuracy H(5) = 22.34, p < 0.05; F1-score H(5) = 13.82, p = 0.0168), there was only a 3% difference in the F1-score between the worst (MobileNet V2) and the best model (Xception). Moreover, the models made similar errors (Adjusted Rand Index (ARI) > 0.88 and Adjusted Mutual Information (AMU) > 0.82). Subsequently, the best model, Xception (Accuracy = 96.1%; F1-score = 0.87; F1-Score = 0.97 with oversampling), was optimized and deployed on the Raspberry Pi, Google Coral, and Nvidia Jetson edge devices using both TenorFlow Lite and TensorRT frameworks. Optimizing the models to run on edge devices reduced the average macro F1-Score to 0.7, and adversely affected the minority classes, reducing their F1-score to as low as 0.18. Upon stress testing, by processing 1000 images consecutively, Jetson Nano, running a TensorRT model, outperformed others with a latency of 0.276 s/image (s.d. = 0.002) while consuming an average current of 1665.21 mA. Raspberry Pi consumed the least average current (838.99 mA) with a ten times worse latency of 2.83 s/image (s.d. = 0.036). Nano was the only reasonable option as an edge device because it could capture most animals whose maximum speeds were below 80 km/h, including goats, lions, ostriches, etc. While the proposed architecture is viable, unbalanced data remain a challenge and the results can potentially be improved by using object detection to reduce imbalances and by exploring semi-supervised learning.American University of SharjahMDPI2026-03-26T08:58:07Z2026-03-26T08:58:07Z2022-01-13Peer-ReviewedPublished versioninfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfZualkernan, I., Dhou, S., Judas, J., Sajun, A. R., Gomez, B. R., & Hussain, L. A. (2022). An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge. Computers, 11(1), 13. https://doi.org/10.3390/computers110100132073-431Xhttps://hdl.handle.net/11073/3326810.3390/computers11010013enhttps://doi.org/10.3390/computers11010013Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/oai:repository.aus.edu:11073/332682026-03-27T05:29:08Z |
| spellingShingle | An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge Zaulkernan, Imran Deep learning Animal classification Image classification Internet of Things Image processing Edge computing Animal surveillance Wildlife monitoring Edge computing |
| status_str | publishedVersion |
| title | An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge |
| title_full | An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge |
| title_fullStr | An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge |
| title_full_unstemmed | An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge |
| title_short | An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge |
| title_sort | An IoT System Using Deep Learning to Classify Camera Trap Images on the Edge |
| topic | Deep learning Animal classification Image classification Internet of Things Image processing Edge computing Animal surveillance Wildlife monitoring Edge computing |
| url | https://hdl.handle.net/11073/33268 |