RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems.
<p dir="ltr">Data analytics are moving beyond the limits of a single platform. In this paper, we present the cost-based optimizer of Rheem, an open-source cross-platform system that copes with these new requirements. The optimizer allocates the subtasks of data analytic tasks to the...
محفوظ في:
| المؤلف الرئيسي: | |
|---|---|
| مؤلفون آخرون: | , , , , |
| منشور في: |
2020
|
| الموضوعات: | |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| _version_ | 1864513514008215552 |
|---|---|
| author | Sebastian Kruse (18595195) |
| author2 | Zoi Kaoudi (14150709) Bertty Contreras-Rojas (18595198) Sanjay Chawla (4254202) Felix Naumann (18595201) Jorge-Arnulfo Quiané-Ruiz (18595204) |
| author2_role | author author author author author |
| author_facet | Sebastian Kruse (18595195) Zoi Kaoudi (14150709) Bertty Contreras-Rojas (18595198) Sanjay Chawla (4254202) Felix Naumann (18595201) Jorge-Arnulfo Quiané-Ruiz (18595204) |
| author_role | author |
| dc.creator.none.fl_str_mv | Sebastian Kruse (18595195) Zoi Kaoudi (14150709) Bertty Contreras-Rojas (18595198) Sanjay Chawla (4254202) Felix Naumann (18595201) Jorge-Arnulfo Quiané-Ruiz (18595204) |
| dc.date.none.fl_str_mv | 2020-05-18T09:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1007/s00778-020-00612-x |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/RHEEMix_in_the_data_jungle_a_cost-based_optimizer_for_cross-platform_systems_/25879486 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Data management and data science Distributed computing and systems software Cross-platform Polystore Query optimization Data processing |
| dc.title.none.fl_str_mv | RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems. |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">Data analytics are moving beyond the limits of a single platform. In this paper, we present the cost-based optimizer of Rheem, an open-source cross-platform system that copes with these new requirements. The optimizer allocates the subtasks of data analytic tasks to the most suitable platforms. Our main contributions are: (i) a mechanism based on graph transformations to explore alternative execution strategies; (ii) a novel graph-based approach to determine efficient data movement plans among subtasks and platforms; and (iii) an efficient plan enumeration algorithm, based on a novel enumeration algebra. We extensively evaluate our optimizer under diverse real tasks. We show that our optimizer can perform tasks more than one order of magnitude faster when using multiple platforms than when using a single platform.</p><h2>Other Information</h2><p dir="ltr">Published in: The VLDB Journal<br>License: <a href="https://creativecommons.org/licenses/by/4.0" target="_blank">https://creativecommons.org/licenses/by/4.0</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1007/s00778-020-00612-x" target="_blank">https://dx.doi.org/10.1007/s00778-020-00612-x</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_6256a8149160eb6d8732f302d7d99cf7 |
| identifier_str_mv | 10.1007/s00778-020-00612-x |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/25879486 |
| publishDate | 2020 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems.Sebastian Kruse (18595195)Zoi Kaoudi (14150709)Bertty Contreras-Rojas (18595198)Sanjay Chawla (4254202)Felix Naumann (18595201)Jorge-Arnulfo Quiané-Ruiz (18595204)Information and computing sciencesData management and data scienceDistributed computing and systems softwareCross-platformPolystoreQuery optimizationData processing<p dir="ltr">Data analytics are moving beyond the limits of a single platform. In this paper, we present the cost-based optimizer of Rheem, an open-source cross-platform system that copes with these new requirements. The optimizer allocates the subtasks of data analytic tasks to the most suitable platforms. Our main contributions are: (i) a mechanism based on graph transformations to explore alternative execution strategies; (ii) a novel graph-based approach to determine efficient data movement plans among subtasks and platforms; and (iii) an efficient plan enumeration algorithm, based on a novel enumeration algebra. We extensively evaluate our optimizer under diverse real tasks. We show that our optimizer can perform tasks more than one order of magnitude faster when using multiple platforms than when using a single platform.</p><h2>Other Information</h2><p dir="ltr">Published in: The VLDB Journal<br>License: <a href="https://creativecommons.org/licenses/by/4.0" target="_blank">https://creativecommons.org/licenses/by/4.0</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1007/s00778-020-00612-x" target="_blank">https://dx.doi.org/10.1007/s00778-020-00612-x</a></p>2020-05-18T09:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1007/s00778-020-00612-xhttps://figshare.com/articles/journal_contribution/RHEEMix_in_the_data_jungle_a_cost-based_optimizer_for_cross-platform_systems_/25879486CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/258794862020-05-18T09:00:00Z |
| spellingShingle | RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems. Sebastian Kruse (18595195) Information and computing sciences Data management and data science Distributed computing and systems software Cross-platform Polystore Query optimization Data processing |
| status_str | publishedVersion |
| title | RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems. |
| title_full | RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems. |
| title_fullStr | RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems. |
| title_full_unstemmed | RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems. |
| title_short | RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems. |
| title_sort | RHEEMix in the data jungle: a cost-based optimizer for cross-platform systems. |
| topic | Information and computing sciences Data management and data science Distributed computing and systems software Cross-platform Polystore Query optimization Data processing |