On the effectiveness of handcrafted features for deepfake video detection
Recent developments in advanced generative deep learning techniques have led to considerable progress in deepfake technology. CNN-based deepfake detection approaches have demonstrated superior performance. The ability to learn meaningful representations generated by convolutional multilayer nonlinea...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , , |
| منشور في: |
2023
|
| الوصول للمادة أونلاين: | https://depot.sorbonne.ae/handle/20.500.12458/1449 |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1857415062795321344 |
|---|---|
| author | Kaddar, Bachir |
| author2 | Fezza, Sid Ahmed Hamidouche, Wassim Akhtar, Zahid Hadid, Abdenour |
| author2_role | author author author author |
| author_facet | Kaddar, Bachir Fezza, Sid Ahmed Hamidouche, Wassim Akhtar, Zahid Hadid, Abdenour |
| author_role | author |
| dc.creator.none.fl_str_mv | Kaddar, Bachir Fezza, Sid Ahmed Hamidouche, Wassim Akhtar, Zahid Hadid, Abdenour |
| dc.date.none.fl_str_mv | 2023-11-30T07:21:00Z 2023-11-30T07:21:00Z 2023 |
| dc.identifier.none.fl_str_mv | 1017-9909 https://depot.sorbonne.ae/handle/20.500.12458/1449 10.1117/1.JEI.32.5.053033 |
| dc.language.none.fl_str_mv | en |
| dc.relation.none.fl_str_mv | Journal of Electronic Imaging |
| dc.title.none.fl_str_mv | On the effectiveness of handcrafted features for deepfake video detection |
| dc.type.none.fl_str_mv | Controlled Vocabulary for Resource Type Genres::text::periodical::journal::contribution to journal::journal article |
| description | Recent developments in advanced generative deep learning techniques have led to considerable progress in deepfake technology. CNN-based deepfake detection approaches have demonstrated superior performance. The ability to learn meaningful representations generated by convolutional multilayer nonlinear structures is the key to success. However, the black-box nature of such approaches has been a major concern for exploring hidden and complex characteristics as well as potential limitations of CNN-based models. To gain insights into the scope of the deepfake detection task, we investigate the effectiveness of handcrafted feature-based methods for deepfake video detection. First, we experiment with six top-performing handcrafted descriptors to extract the discriminating image features and then train SVMs on the extracted features to learn a suitable model. We also study the effect of selecting specific facial components on the detection performance. Specifically, we consider features extracted from the left eye, right eye, mouth, and entire face. Moreover, we propose a combination of these features and highlight the importance of this combination in terms of detection performance. Experimental results show that the SIFT feature descriptor achieves the best performance on deepfake videos generated by the neural texture technique, with a detection accuracy of 83.50%, which is better than deep learning-based methods. This is in contrast to the conventional understanding that deep learning methods systematically outperform handcrafted feature-based approaches. In addition, the obtained results on the FaceForensics++ dataset highlight the benefit of using some facial components to further boost the detection performance. Moreover, motivated by the effectiveness of the LBPTOP and SIFT in the deepfake detection task, we combined the LBPTOP and SIFT to best characterize the specific spatiotemporal inconsistencies commonly found in fake videos for boosting deepfake detection performance. Finally, we show the strengths and weaknesses of methods based on handcrafted features for deepfake detection and provide directions for future research. |
| id | sorbonner_3692adfc5791622332f245f4c6a8a032 |
| identifier_str_mv | 1017-9909 10.1117/1.JEI.32.5.053033 |
| language_invalid_str_mv | en |
| network_acronym_str | sorbonner |
| network_name_str | Sorbonne University Abu Dhabi repository |
| oai_identifier_str | oai:depot.sorbonne.ae:20.500.12458/1449 |
| publishDate | 2023 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| spelling | On the effectiveness of handcrafted features for deepfake video detectionKaddar, BachirFezza, Sid AhmedHamidouche, WassimAkhtar, ZahidHadid, AbdenourRecent developments in advanced generative deep learning techniques have led to considerable progress in deepfake technology. CNN-based deepfake detection approaches have demonstrated superior performance. The ability to learn meaningful representations generated by convolutional multilayer nonlinear structures is the key to success. However, the black-box nature of such approaches has been a major concern for exploring hidden and complex characteristics as well as potential limitations of CNN-based models. To gain insights into the scope of the deepfake detection task, we investigate the effectiveness of handcrafted feature-based methods for deepfake video detection. First, we experiment with six top-performing handcrafted descriptors to extract the discriminating image features and then train SVMs on the extracted features to learn a suitable model. We also study the effect of selecting specific facial components on the detection performance. Specifically, we consider features extracted from the left eye, right eye, mouth, and entire face. Moreover, we propose a combination of these features and highlight the importance of this combination in terms of detection performance. Experimental results show that the SIFT feature descriptor achieves the best performance on deepfake videos generated by the neural texture technique, with a detection accuracy of 83.50%, which is better than deep learning-based methods. This is in contrast to the conventional understanding that deep learning methods systematically outperform handcrafted feature-based approaches. In addition, the obtained results on the FaceForensics++ dataset highlight the benefit of using some facial components to further boost the detection performance. Moreover, motivated by the effectiveness of the LBPTOP and SIFT in the deepfake detection task, we combined the LBPTOP and SIFT to best characterize the specific spatiotemporal inconsistencies commonly found in fake videos for boosting deepfake detection performance. Finally, we show the strengths and weaknesses of methods based on handcrafted features for deepfake detection and provide directions for future research.2023-11-30T07:21:00Z2023-11-30T07:21:00Z2023Controlled Vocabulary for Resource Type Genres::text::periodical::journal::contribution to journal::journal article1017-9909https://depot.sorbonne.ae/handle/20.500.12458/144910.1117/1.JEI.32.5.053033enJournal of Electronic Imagingoai:depot.sorbonne.ae:20.500.12458/14492023-11-30T07:21:00Z |
| spellingShingle | On the effectiveness of handcrafted features for deepfake video detection Kaddar, Bachir |
| title | On the effectiveness of handcrafted features for deepfake video detection |
| title_full | On the effectiveness of handcrafted features for deepfake video detection |
| title_fullStr | On the effectiveness of handcrafted features for deepfake video detection |
| title_full_unstemmed | On the effectiveness of handcrafted features for deepfake video detection |
| title_short | On the effectiveness of handcrafted features for deepfake video detection |
| title_sort | On the effectiveness of handcrafted features for deepfake video detection |
| url | https://depot.sorbonne.ae/handle/20.500.12458/1449 |