CNN and HEVC Video Coding Features for Static Video Summarization
This study proposes a novel solution for the detection of keyframes for static video summarization. We preprocessed the well-known video datasets by coding them using the HEVC video coding standard. During coding, 64 proposed features were generated from the coder for each frame. Additionally, we co...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | |
| التنسيق: | article |
| منشور في: |
2022
|
| الموضوعات: | |
| الوصول للمادة أونلاين: | http://hdl.handle.net/11073/24062 |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513434517766144 |
|---|---|
| author | Issa, Obada |
| author2 | Shanableh, Tamer |
| author2_role | author |
| author_facet | Issa, Obada Shanableh, Tamer |
| author_role | author |
| dc.creator.none.fl_str_mv | Issa, Obada Shanableh, Tamer |
| dc.date.none.fl_str_mv | 2022-07-07T07:27:39Z 2022-07-07T07:27:39Z 2022 |
| dc.format.none.fl_str_mv | application/pdf |
| dc.identifier.none.fl_str_mv | O. Issa and T. Shanableh, "CNN and HEVC Video Coding Features for Static Video Summarization," in IEEE Access, 2022, doi: 10.1109/ACCESS.2022.3188638. 2169-3536 http://hdl.handle.net/11073/24062 |
| dc.language.none.fl_str_mv | en_US |
| dc.publisher.none.fl_str_mv | IEEE |
| dc.relation.none.fl_str_mv | https://doi.org/10.1109/ACCESS.2022.3188638 |
| dc.subject.none.fl_str_mv | Convolution neural network Duplicate frames Sparse auto encoders Static video Summarization Video coding HEVC (High Efficiency Video Codec) |
| dc.title.none.fl_str_mv | CNN and HEVC Video Coding Features for Static Video Summarization |
| dc.type.none.fl_str_mv | Peer-Reviewed Published version info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/article |
| description | This study proposes a novel solution for the detection of keyframes for static video summarization. We preprocessed the well-known video datasets by coding them using the HEVC video coding standard. During coding, 64 proposed features were generated from the coder for each frame. Additionally, we converted the original YUVs of the raw videos into RGB images and fed them into pretrained CNN networks for feature extraction. These include GoogleNet, AlexNet, Inception-ResNet-v2, and VGG16. The modified datasets are made publicly available to the research community. Before detecting keyframes in a video, it is important to identify and eliminate duplicate or similar video frames. A subset of the proposed HEVC feature set was used to identify these frames and eliminate them from the video. We also propose an elimination solution based on the sum of the absolute differences between a frame and its motion-compensated predecessor. The proposed solutions are compared with existing works based on an SIFT flow algorithm that uses CNN features. Subsequently, an optional dimensionality reduction based on stepwise regression was applied to the feature vectors prior to detecting key frames. The proposed solution is compared with existing studies that use sparse autoencoders with CNN features for dimensionality reduction. The accuracy of the proposed key-frame detection system was assessed using the positive predictive values, sensitivity, and F-scores. Combining the proposed solution with Multi-CNN features and using a random forest classifier, it was shown that the proposed solution achieved an average F-score of 0.98. |
| format | article |
| id | aus_9faa2598904e365052d8e9e86171ab54 |
| identifier_str_mv | O. Issa and T. Shanableh, "CNN and HEVC Video Coding Features for Static Video Summarization," in IEEE Access, 2022, doi: 10.1109/ACCESS.2022.3188638. 2169-3536 |
| language_invalid_str_mv | en_US |
| network_acronym_str | aus |
| network_name_str | aus |
| oai_identifier_str | oai:repository.aus.edu:11073/24062 |
| publishDate | 2022 |
| publisher.none.fl_str_mv | IEEE |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| spelling | CNN and HEVC Video Coding Features for Static Video SummarizationIssa, ObadaShanableh, TamerConvolution neural networkDuplicate framesSparse auto encodersStatic videoSummarizationVideo codingHEVC (High Efficiency Video Codec)This study proposes a novel solution for the detection of keyframes for static video summarization. We preprocessed the well-known video datasets by coding them using the HEVC video coding standard. During coding, 64 proposed features were generated from the coder for each frame. Additionally, we converted the original YUVs of the raw videos into RGB images and fed them into pretrained CNN networks for feature extraction. These include GoogleNet, AlexNet, Inception-ResNet-v2, and VGG16. The modified datasets are made publicly available to the research community. Before detecting keyframes in a video, it is important to identify and eliminate duplicate or similar video frames. A subset of the proposed HEVC feature set was used to identify these frames and eliminate them from the video. We also propose an elimination solution based on the sum of the absolute differences between a frame and its motion-compensated predecessor. The proposed solutions are compared with existing works based on an SIFT flow algorithm that uses CNN features. Subsequently, an optional dimensionality reduction based on stepwise regression was applied to the feature vectors prior to detecting key frames. The proposed solution is compared with existing studies that use sparse autoencoders with CNN features for dimensionality reduction. The accuracy of the proposed key-frame detection system was assessed using the positive predictive values, sensitivity, and F-scores. Combining the proposed solution with Multi-CNN features and using a random forest classifier, it was shown that the proposed solution achieved an average F-score of 0.98.American University of SharjahIEEE2022-07-07T07:27:39Z2022-07-07T07:27:39Z2022Peer-ReviewedPublished versioninfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfO. Issa and T. Shanableh, "CNN and HEVC Video Coding Features for Static Video Summarization," in IEEE Access, 2022, doi: 10.1109/ACCESS.2022.3188638.2169-3536http://hdl.handle.net/11073/24062en_UShttps://doi.org/10.1109/ACCESS.2022.3188638oai:repository.aus.edu:11073/240622024-08-22T12:07:48Z |
| spellingShingle | CNN and HEVC Video Coding Features for Static Video Summarization Issa, Obada Convolution neural network Duplicate frames Sparse auto encoders Static video Summarization Video coding HEVC (High Efficiency Video Codec) |
| status_str | publishedVersion |
| title | CNN and HEVC Video Coding Features for Static Video Summarization |
| title_full | CNN and HEVC Video Coding Features for Static Video Summarization |
| title_fullStr | CNN and HEVC Video Coding Features for Static Video Summarization |
| title_full_unstemmed | CNN and HEVC Video Coding Features for Static Video Summarization |
| title_short | CNN and HEVC Video Coding Features for Static Video Summarization |
| title_sort | CNN and HEVC Video Coding Features for Static Video Summarization |
| topic | Convolution neural network Duplicate frames Sparse auto encoders Static video Summarization Video coding HEVC (High Efficiency Video Codec) |
| url | http://hdl.handle.net/11073/24062 |