Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging
<p dir="ltr">Word segmentation plays a pivotal role in improving any Arabic NLP application. Therefore, a lot of research has been spent in improving its accuracy. Off-the-shelf tools, however, are: i) complicated to use and ii) domain/dialect dependent. We explore three language-ind...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , , , |
| منشور في: |
2017
|
| الموضوعات: | |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513557097349120 |
|---|---|
| author | Hassan Sajjad (5297441) |
| author2 | Fahim Dalvi (18427905) Nadir Durrani (5297438) Ahmed Abdelali (19691659) Yonatan Belinkov (18973897) Stephan Vogel (19691698) |
| author2_role | author author author author author |
| author_facet | Hassan Sajjad (5297441) Fahim Dalvi (18427905) Nadir Durrani (5297438) Ahmed Abdelali (19691659) Yonatan Belinkov (18973897) Stephan Vogel (19691698) |
| author_role | author |
| dc.creator.none.fl_str_mv | Hassan Sajjad (5297441) Fahim Dalvi (18427905) Nadir Durrani (5297438) Ahmed Abdelali (19691659) Yonatan Belinkov (18973897) Stephan Vogel (19691698) |
| dc.date.none.fl_str_mv | 2017-07-30T06:00:00Z |
| dc.identifier.none.fl_str_mv | 10.18653/v1/p17-2095 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/conference_contribution/Challenging_Language-Dependent_Segmentation_for_Arabic_An__Application_to_Machine_Translation_and_Part-of-Speech_Tagging/27050737 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Artificial intelligence Language, communication and culture Linguistics Word Segmentation Arabic NLP Morphological Segmentation Accuracy Improvement Off-the-Shelf Tools Domain Dependency Dialect Dependency |
| dc.title.none.fl_str_mv | Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging |
| dc.type.none.fl_str_mv | Text Conference contribution info:eu-repo/semantics/publishedVersion text conference object |
| description | <p dir="ltr">Word segmentation plays a pivotal role in improving any Arabic NLP application. Therefore, a lot of research has been spent in improving its accuracy. Off-the-shelf tools, however, are: i) complicated to use and ii) domain/dialect dependent. We explore three language-independent alternatives to morphological segmentation using: i) data-driven sub-word units, ii) characters as a unit of learning, and iii) word embeddings learned using a character CNN (Convolution Neural Network). On the tasks of Machine Translation and POS tagging, we found these methods to achieve close to, and occasionally surpass state-of-the-art performance. In our analysis, we show that a neural machine translation system is sensitive to the ratio of source and target tokens, and a ratio close to 1 or greater, gives optimal performance.</p><h2>Other Information</h2><p dir="ltr">Published in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See conference contribution on publisher's website: <a href="https://dx.doi.org/10.18653/v1/p17-2095" target="_blank">https://dx.doi.org/10.18653/v1/p17-2095</a></p><p dir="ltr">Conference information: 55th Annual Meeting of the Association for Computational Linguistics (Short Papers), pages 518–523 Vancouver, Canada, July 30 - August 4, 2017</p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_ab19b804bae19ef57dbc15c1615860c8 |
| identifier_str_mv | 10.18653/v1/p17-2095 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/27050737 |
| publishDate | 2017 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech TaggingHassan Sajjad (5297441)Fahim Dalvi (18427905)Nadir Durrani (5297438)Ahmed Abdelali (19691659)Yonatan Belinkov (18973897)Stephan Vogel (19691698)Information and computing sciencesArtificial intelligenceLanguage, communication and cultureLinguisticsWord SegmentationArabic NLPMorphological SegmentationAccuracy ImprovementOff-the-Shelf ToolsDomain DependencyDialect Dependency<p dir="ltr">Word segmentation plays a pivotal role in improving any Arabic NLP application. Therefore, a lot of research has been spent in improving its accuracy. Off-the-shelf tools, however, are: i) complicated to use and ii) domain/dialect dependent. We explore three language-independent alternatives to morphological segmentation using: i) data-driven sub-word units, ii) characters as a unit of learning, and iii) word embeddings learned using a character CNN (Convolution Neural Network). On the tasks of Machine Translation and POS tagging, we found these methods to achieve close to, and occasionally surpass state-of-the-art performance. In our analysis, we show that a neural machine translation system is sensitive to the ratio of source and target tokens, and a ratio close to 1 or greater, gives optimal performance.</p><h2>Other Information</h2><p dir="ltr">Published in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See conference contribution on publisher's website: <a href="https://dx.doi.org/10.18653/v1/p17-2095" target="_blank">https://dx.doi.org/10.18653/v1/p17-2095</a></p><p dir="ltr">Conference information: 55th Annual Meeting of the Association for Computational Linguistics (Short Papers), pages 518–523 Vancouver, Canada, July 30 - August 4, 2017</p>2017-07-30T06:00:00ZTextConference contributioninfo:eu-repo/semantics/publishedVersiontextconference object10.18653/v1/p17-2095https://figshare.com/articles/conference_contribution/Challenging_Language-Dependent_Segmentation_for_Arabic_An__Application_to_Machine_Translation_and_Part-of-Speech_Tagging/27050737CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/270507372017-07-30T06:00:00Z |
| spellingShingle | Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging Hassan Sajjad (5297441) Information and computing sciences Artificial intelligence Language, communication and culture Linguistics Word Segmentation Arabic NLP Morphological Segmentation Accuracy Improvement Off-the-Shelf Tools Domain Dependency Dialect Dependency |
| status_str | publishedVersion |
| title | Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging |
| title_full | Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging |
| title_fullStr | Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging |
| title_full_unstemmed | Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging |
| title_short | Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging |
| title_sort | Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging |
| topic | Information and computing sciences Artificial intelligence Language, communication and culture Linguistics Word Segmentation Arabic NLP Morphological Segmentation Accuracy Improvement Off-the-Shelf Tools Domain Dependency Dialect Dependency |