Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning
There is an abundance of digital video content due to the cloud’s phenomenal growth and security footage, it is therefore essential to summarize these videos in data centers. This paper offers innovative approaches to the problem of key-frame extraction for the purpose of video summarization. Our ap...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | article |
| Published: |
2023
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/11073/25249 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513432890376192 |
|---|---|
| author | Issa, Obada |
| author2 | Shanableh, Tamer |
| author2_role | author |
| author_facet | Issa, Obada Shanableh, Tamer |
| author_role | author |
| dc.creator.none.fl_str_mv | Issa, Obada Shanableh, Tamer |
| dc.date.none.fl_str_mv | 2023-05-15T09:55:30Z 2023-05-15T09:55:30Z 2023 |
| dc.format.none.fl_str_mv | application/pdf |
| dc.identifier.none.fl_str_mv | Issa O, Shanableh T. Static Video Summarization Using Video Coding Features with Frame-Level Temporal Subsampling and Deep Learning. Applied Sciences. 2023; 13(10):6065. 2076-3417 http://hdl.handle.net/11073/25249 10.3390/app13106065 |
| dc.language.none.fl_str_mv | en_US |
| dc.publisher.none.fl_str_mv | MDPI |
| dc.relation.none.fl_str_mv | https://doi.org/10.3390/app13106065 |
| dc.subject.none.fl_str_mv | Video summarization Video coding Temporal subsampling Convolution neural networks Long-short term memory |
| dc.title.none.fl_str_mv | Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning |
| dc.type.none.fl_str_mv | Peer-Reviewed Postprint info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/article |
| description | There is an abundance of digital video content due to the cloud’s phenomenal growth and security footage, it is therefore essential to summarize these videos in data centers. This paper offers innovative approaches to the problem of key-frame extraction for the purpose of video summarization. Our approach includes feature variables extracted from the bit streams of coded videos, followed by optional stepwise regression for dimensionality reduction. Once the features are extracted and reduced in dimensionality, we apply innovate frame-level temporal sub-sampling techniques followed by training and testing using deep learning architectures. The frame-level temporal subsampling techniques are based on cosine similarity and PCA projections of feature vectors. We create three different learning architectures by utilizing LSTM networks, 1D-CNN networks, and Random Forests. The four most popular video summarization datasets, namely, TVSum, SumMe, OVP, and VSUMM are used to evaluate the accuracy of the proposed solutions. This includes the Precision, Recall, F-score measures, and computational time. It is shown that the proposed solutions when trained and tested on all subjective user summaries, achieved F-scores of 0.79, 0.74, 0.88, and 0.81, respectively, for the aforementioned datasets, showing clear improvements over prior studies. |
| format | article |
| id | aus_f8c14f1a3a63b53761fd9fc461288bfd |
| identifier_str_mv | Issa O, Shanableh T. Static Video Summarization Using Video Coding Features with Frame-Level Temporal Subsampling and Deep Learning. Applied Sciences. 2023; 13(10):6065. 2076-3417 10.3390/app13106065 |
| language_invalid_str_mv | en_US |
| network_acronym_str | aus |
| network_name_str | aus |
| oai_identifier_str | oai:repository.aus.edu:11073/25249 |
| publishDate | 2023 |
| publisher.none.fl_str_mv | MDPI |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| spelling | Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep LearningIssa, ObadaShanableh, TamerVideo summarizationVideo codingTemporal subsamplingConvolution neural networksLong-short term memoryThere is an abundance of digital video content due to the cloud’s phenomenal growth and security footage, it is therefore essential to summarize these videos in data centers. This paper offers innovative approaches to the problem of key-frame extraction for the purpose of video summarization. Our approach includes feature variables extracted from the bit streams of coded videos, followed by optional stepwise regression for dimensionality reduction. Once the features are extracted and reduced in dimensionality, we apply innovate frame-level temporal sub-sampling techniques followed by training and testing using deep learning architectures. The frame-level temporal subsampling techniques are based on cosine similarity and PCA projections of feature vectors. We create three different learning architectures by utilizing LSTM networks, 1D-CNN networks, and Random Forests. The four most popular video summarization datasets, namely, TVSum, SumMe, OVP, and VSUMM are used to evaluate the accuracy of the proposed solutions. This includes the Precision, Recall, F-score measures, and computational time. It is shown that the proposed solutions when trained and tested on all subjective user summaries, achieved F-scores of 0.79, 0.74, 0.88, and 0.81, respectively, for the aforementioned datasets, showing clear improvements over prior studies.American University of SharjahMDPI2023-05-15T09:55:30Z2023-05-15T09:55:30Z2023Peer-ReviewedPostprintinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfIssa O, Shanableh T. Static Video Summarization Using Video Coding Features with Frame-Level Temporal Subsampling and Deep Learning. Applied Sciences. 2023; 13(10):6065.2076-3417http://hdl.handle.net/11073/2524910.3390/app13106065en_UShttps://doi.org/10.3390/app13106065oai:repository.aus.edu:11073/252492024-08-22T12:07:18Z |
| spellingShingle | Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning Issa, Obada Video summarization Video coding Temporal subsampling Convolution neural networks Long-short term memory |
| status_str | publishedVersion |
| title | Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning |
| title_full | Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning |
| title_fullStr | Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning |
| title_full_unstemmed | Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning |
| title_short | Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning |
| title_sort | Static Video Summarization Using Video Coding Features with Frame-level Temporal Sub-Sampling and Deep Learning |
| topic | Video summarization Video coding Temporal subsampling Convolution neural networks Long-short term memory |
| url | http://hdl.handle.net/11073/25249 |