Self-Crowdsourcing Training for Relation Extraction
<p dir="ltr">One expensive step when defining crowdsourcing tasks is to define the examples and control questions for instructing the crowd workers. In this paper, we introduce a self-training strategy for crowdsourcing. The main idea is to use an automatic classifier, trained on wea...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , |
| Published: |
2017
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513557095251968 |
|---|---|
| author | Azad Abad (19691692) |
| author2 | Moin Nabi (19691695) Alessandro Moschitti (19691683) |
| author2_role | author author |
| author_facet | Azad Abad (19691692) Moin Nabi (19691695) Alessandro Moschitti (19691683) |
| author_role | author |
| dc.creator.none.fl_str_mv | Azad Abad (19691692) Moin Nabi (19691695) Alessandro Moschitti (19691683) |
| dc.date.none.fl_str_mv | 2017-07-01T03:00:00Z |
| dc.identifier.none.fl_str_mv | 10.18653/v1/p17-2082 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/conference_contribution/Self-Crowdsourcing_Training_for_Relation_Extraction/27050734 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Artificial intelligence Human-centred computing Language, communication and culture Linguistics Crowdsourcing Self-training strategy Automatic classifier Weakly supervised data High confidence examples |
| dc.title.none.fl_str_mv | Self-Crowdsourcing Training for Relation Extraction |
| dc.type.none.fl_str_mv | Text Conference contribution info:eu-repo/semantics/publishedVersion text conference object |
| description | <p dir="ltr">One expensive step when defining crowdsourcing tasks is to define the examples and control questions for instructing the crowd workers. In this paper, we introduce a self-training strategy for crowdsourcing. The main idea is to use an automatic classifier, trained on weakly supervised data, to select examples associated with high confidence. These are used by our automatic agent to explain the task to crowd workers with a question answering approach. We compared our relation extraction system trained with data annotated (i) with distant supervision and (ii) by workers instructed with our approach. The analysis shows that our method relatively improves the relation extraction system by about 11% in F1.</p><h2>Other Information</h2><p dir="ltr">Published in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See conference contribution on publisher's website: <a href="https://dx.doi.org/10.18653/v1/p17-2082" target="_blank">https://dx.doi.org/10.18653/v1/p17-2082</a></p><p dir="ltr">Conference information: 55th Annual Meeting of the Association for Computational Linguistics (Short Papers), pages 518–523 Vancouver, Canada, July 30 - August 4, 2017</p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_337e78487321026373ca41b897e85f8f |
| identifier_str_mv | 10.18653/v1/p17-2082 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/27050734 |
| publishDate | 2017 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Self-Crowdsourcing Training for Relation ExtractionAzad Abad (19691692)Moin Nabi (19691695)Alessandro Moschitti (19691683)Information and computing sciencesArtificial intelligenceHuman-centred computingLanguage, communication and cultureLinguisticsCrowdsourcingSelf-training strategyAutomatic classifierWeakly supervised dataHigh confidence examples<p dir="ltr">One expensive step when defining crowdsourcing tasks is to define the examples and control questions for instructing the crowd workers. In this paper, we introduce a self-training strategy for crowdsourcing. The main idea is to use an automatic classifier, trained on weakly supervised data, to select examples associated with high confidence. These are used by our automatic agent to explain the task to crowd workers with a question answering approach. We compared our relation extraction system trained with data annotated (i) with distant supervision and (ii) by workers instructed with our approach. The analysis shows that our method relatively improves the relation extraction system by about 11% in F1.</p><h2>Other Information</h2><p dir="ltr">Published in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)<br>License: <a href="http://creativecommons.org/licenses/by/4.0/" target="_blank">http://creativecommons.org/licenses/by/4.0/</a><br>See conference contribution on publisher's website: <a href="https://dx.doi.org/10.18653/v1/p17-2082" target="_blank">https://dx.doi.org/10.18653/v1/p17-2082</a></p><p dir="ltr">Conference information: 55th Annual Meeting of the Association for Computational Linguistics (Short Papers), pages 518–523 Vancouver, Canada, July 30 - August 4, 2017</p>2017-07-01T03:00:00ZTextConference contributioninfo:eu-repo/semantics/publishedVersiontextconference object10.18653/v1/p17-2082https://figshare.com/articles/conference_contribution/Self-Crowdsourcing_Training_for_Relation_Extraction/27050734CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/270507342017-07-01T03:00:00Z |
| spellingShingle | Self-Crowdsourcing Training for Relation Extraction Azad Abad (19691692) Information and computing sciences Artificial intelligence Human-centred computing Language, communication and culture Linguistics Crowdsourcing Self-training strategy Automatic classifier Weakly supervised data High confidence examples |
| status_str | publishedVersion |
| title | Self-Crowdsourcing Training for Relation Extraction |
| title_full | Self-Crowdsourcing Training for Relation Extraction |
| title_fullStr | Self-Crowdsourcing Training for Relation Extraction |
| title_full_unstemmed | Self-Crowdsourcing Training for Relation Extraction |
| title_short | Self-Crowdsourcing Training for Relation Extraction |
| title_sort | Self-Crowdsourcing Training for Relation Extraction |
| topic | Information and computing sciences Artificial intelligence Human-centred computing Language, communication and culture Linguistics Crowdsourcing Self-training strategy Automatic classifier Weakly supervised data High confidence examples |