Generation and Detection of Sign Language Deepfakes
A Master of Science thesis in Electrical Engineering by Shahzeb Naeem entitled, “Generation and Detection of Sign Language Deepfakes”, submitted in December 2024. Thesis advisor is Dr. Usman Tariq and thesis co-advisors are Dr. Hasan Al-Nashash and Dr. Abhinav Dhall. Soft copy is available (Thesis,...
Saved in:
| Main Author: | |
|---|---|
| Format: | doctoralThesis |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/11073/25782 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513439324438528 |
|---|---|
| author | Naeem, Shahzeb |
| author_facet | Naeem, Shahzeb |
| author_role | author |
| dc.contributor.none.fl_str_mv | Tariq, Usman Al-Nashash, Hasan Dhall, Abhinav |
| dc.creator.none.fl_str_mv | Naeem, Shahzeb |
| dc.date.none.fl_str_mv | 2024-12 2025-01-21T07:42:48Z 2025-01-21T07:42:48Z |
| dc.format.none.fl_str_mv | application/pdf application/pdf |
| dc.identifier.none.fl_str_mv | 35.232-2024.49 https://hdl.handle.net/11073/25782 |
| dc.language.none.fl_str_mv | en_US |
| dc.subject.none.fl_str_mv | Deepfakes, Artificial intelligence Sign language deepfake |
| dc.title.none.fl_str_mv | Generation and Detection of Sign Language Deepfakes |
| dc.type.none.fl_str_mv | info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/doctoralThesis |
| description | A Master of Science thesis in Electrical Engineering by Shahzeb Naeem entitled, “Generation and Detection of Sign Language Deepfakes”, submitted in December 2024. Thesis advisor is Dr. Usman Tariq and thesis co-advisors are Dr. Hasan Al-Nashash and Dr. Abhinav Dhall. Soft copy is available (Thesis, Completion Certificate, Approval Signatures, and AUS Archives Consent Form). |
| format | doctoralThesis |
| id | aus_da984a24ea8cbd3e49262a32159ae76e |
| identifier_str_mv | 35.232-2024.49 |
| language_invalid_str_mv | en_US |
| network_acronym_str | aus |
| network_name_str | aus |
| oai_identifier_str | oai:repository.aus.edu:11073/25782 |
| publishDate | 2024 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| spelling | Generation and Detection of Sign Language DeepfakesNaeem, ShahzebDeepfakes,Artificial intelligenceSign language deepfakeA Master of Science thesis in Electrical Engineering by Shahzeb Naeem entitled, “Generation and Detection of Sign Language Deepfakes”, submitted in December 2024. Thesis advisor is Dr. Usman Tariq and thesis co-advisors are Dr. Hasan Al-Nashash and Dr. Abhinav Dhall. Soft copy is available (Thesis, Completion Certificate, Approval Signatures, and AUS Archives Consent Form).The emergence of synthetic data, or "Deepfakes," in the age of sophisticated visual effects and artificial intelligence has raised questions about potential harm and deception. In contrast, this study investigates the benefits of deepfake technology with a particular emphasis on helping the Deaf and Hard of Hearing (DHoH) community. The reasons behind the lack of such work not having been done before are the complexities of sign language and the scarcity of sign language experts. The objectives of this thesis are to develop a generative model for generating deepfakes in sign language while producing a sign language deepfake dataset that is technically credible and visually convincing using expert analysis. The inputs to the generative model are a source image, and a driving video. The deepfake output is essentially an identity transfer of the source image onto the driving video. The thesis also explores sign language deepfake detection using traditional Machine Learning and Deep Learning models from an unconventional angle using a series of extensive experiments and human interaction after studying real, fake and synthetic images in depth. The analysis of 1200 videos, including unseen persons, reveals a deepfake dataset for assessing model performance. Linguistic analysis, which uses textual similarity scores and an interpreter's evaluation, shows promise in distinguishing between authentic and fraudulent sign language recordings. Even with totally unseen participants, it is possible to produce visually convincing deepfake videos using our approach. It is also possible to detect such deepfakes using much simpler models than we have come to know and expect. The thesis is structured with a literature review, methodology, thorough analysis, findings/results, and discussions. The accuracy of 83.3% by the expert and metric scores close to 1 point to the possibility of using deepfake technology to produce convincing and accurate sign language videos, which would help the DHoH community's inclusivity and education. They also showcase the potential of moving towards more efficient models for deepfake detection and the level of plausibility we have reached in producing images from a computer only or using deepfakes.College of EngineeringDepartment of Electrical EngineeringMaster of Science in Electrical Engineering (MSEE)Tariq, UsmanAl-Nashash, HasanDhall, Abhinav2025-01-21T07:42:48Z2025-01-21T07:42:48Z2024-12info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/doctoralThesisapplication/pdfapplication/pdf35.232-2024.49https://hdl.handle.net/11073/25782en_USoai:repository.aus.edu:11073/257822025-06-26T12:20:31Z |
| spellingShingle | Generation and Detection of Sign Language Deepfakes Naeem, Shahzeb Deepfakes, Artificial intelligence Sign language deepfake |
| status_str | publishedVersion |
| title | Generation and Detection of Sign Language Deepfakes |
| title_full | Generation and Detection of Sign Language Deepfakes |
| title_fullStr | Generation and Detection of Sign Language Deepfakes |
| title_full_unstemmed | Generation and Detection of Sign Language Deepfakes |
| title_short | Generation and Detection of Sign Language Deepfakes |
| title_sort | Generation and Detection of Sign Language Deepfakes |
| topic | Deepfakes, Artificial intelligence Sign language deepfake |
| url | https://hdl.handle.net/11073/25782 |