Arabic Multimodal Emotion Recognition Using Deep Learning
A Master of Science thesis in Computer Engineering by Noora Mohammed Abdulla Al Roken entitled, “Arabic Multimodal Emotion Recognition Using Deep Learning”, submitted in May 2022. Thesis advisors are Dr. Gerassimos Barlas and Dr. Osameh Mahmoud Al-Kofahi. Soft copy is available (Thesis, Completion C...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| التنسيق: | doctoralThesis |
| منشور في: |
2022
|
| الموضوعات: | |
| الوصول للمادة أونلاين: | http://hdl.handle.net/11073/24089 |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513444635475968 |
|---|---|
| author | Al Roken, Noora Mohammed Abdulla |
| author_facet | Al Roken, Noora Mohammed Abdulla |
| author_role | author |
| dc.contributor.none.fl_str_mv | Barlas, Gerassimos Al-Kofahi, Osameh Mahmoud |
| dc.creator.none.fl_str_mv | Al Roken, Noora Mohammed Abdulla |
| dc.date.none.fl_str_mv | 2022-09-06T06:01:42Z 2022-09-06T06:01:42Z 2022-05 |
| dc.format.none.fl_str_mv | application/pdf application/pdf |
| dc.identifier.none.fl_str_mv | 35.232-2022.09 http://hdl.handle.net/11073/24089 |
| dc.language.none.fl_str_mv | en_US |
| dc.subject.none.fl_str_mv | Emotion Recognition (ER) Multimodal ER Deep learning Human computer interaction |
| dc.title.none.fl_str_mv | Arabic Multimodal Emotion Recognition Using Deep Learning |
| dc.type.none.fl_str_mv | info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/doctoralThesis |
| description | A Master of Science thesis in Computer Engineering by Noora Mohammed Abdulla Al Roken entitled, “Arabic Multimodal Emotion Recognition Using Deep Learning”, submitted in May 2022. Thesis advisors are Dr. Gerassimos Barlas and Dr. Osameh Mahmoud Al-Kofahi. Soft copy is available (Thesis, Completion Certificate, Approval Signatures, and AUS Archives Consent Form). |
| format | doctoralThesis |
| id | aus_cf4bb0e51b5e84c38c262d536baffa9a |
| identifier_str_mv | 35.232-2022.09 |
| language_invalid_str_mv | en_US |
| network_acronym_str | aus |
| network_name_str | aus |
| oai_identifier_str | oai:repository.aus.edu:11073/24089 |
| publishDate | 2022 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| spelling | Arabic Multimodal Emotion Recognition Using Deep LearningAl Roken, Noora Mohammed AbdullaEmotion Recognition (ER)Multimodal ERDeep learningHuman computer interactionA Master of Science thesis in Computer Engineering by Noora Mohammed Abdulla Al Roken entitled, “Arabic Multimodal Emotion Recognition Using Deep Learning”, submitted in May 2022. Thesis advisors are Dr. Gerassimos Barlas and Dr. Osameh Mahmoud Al-Kofahi. Soft copy is available (Thesis, Completion Certificate, Approval Signatures, and AUS Archives Consent Form).Emotions are an essential part of human communication since it shapes how information is received. As part of human-computer interaction, researchers are extending Emotion Recognition (ER) to machines. ER can contribute to many fields such as business, education, psychology, and psychiatry due to the importance of emotional insights. ER has been an active area for decades due to the complexity of the problem. Some methods use speech to extract emotions, and others use facial expressions or text. Recently, works began to combine multiple inputs, or modalities, to extract valuable and accurate insights since different emotions might be presented better in different modalities. Many datasets were made available, especially for the speech modality. Different dataset types were used that include the acted, elicited, and natural, and they were built using different languages, including Arabic. However, works utilizing multiple modalities were mainly focused on western and south Asian countries, and none included Arabic. The classifiers built were all trained on acted datasets that include exaggerated reactions, which cannot be reliable. Therefore, in this thesis we present our Arabic audio-visual emotion dataset built on five basic emotions using natural responses. We implement three existing multimodal classifiers and our proposed classifier on our dataset using five-fold cross-validation. Finally, we evaluated the ER performance based on the visual dataset size, joint and disjoint training, and the single and multimodal networks. The performance of the proposed classifier gave the highest average F1-score of 0.504 and an accuracy of 54.88% for natural emotion recognition.College of EngineeringDepartment of Computer Science and EngineeringMaster of Science in Computer Engineering (MSCoE)Barlas, GerassimosAl-Kofahi, Osameh Mahmoud2022-09-06T06:01:42Z2022-09-06T06:01:42Z2022-05info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/doctoralThesisapplication/pdfapplication/pdf35.232-2022.09http://hdl.handle.net/11073/24089en_USoai:repository.aus.edu:11073/240892025-06-26T12:36:33Z |
| spellingShingle | Arabic Multimodal Emotion Recognition Using Deep Learning Al Roken, Noora Mohammed Abdulla Emotion Recognition (ER) Multimodal ER Deep learning Human computer interaction |
| status_str | publishedVersion |
| title | Arabic Multimodal Emotion Recognition Using Deep Learning |
| title_full | Arabic Multimodal Emotion Recognition Using Deep Learning |
| title_fullStr | Arabic Multimodal Emotion Recognition Using Deep Learning |
| title_full_unstemmed | Arabic Multimodal Emotion Recognition Using Deep Learning |
| title_short | Arabic Multimodal Emotion Recognition Using Deep Learning |
| title_sort | Arabic Multimodal Emotion Recognition Using Deep Learning |
| topic | Emotion Recognition (ER) Multimodal ER Deep learning Human computer interaction |
| url | http://hdl.handle.net/11073/24089 |