Szczegóły publikacji

Opis bibliograficzny

Scientific workflow management on hybrid clouds with cloud bursting and transparent data access / Bartosz BALIŚ, Michał ORZECHOWSKI, Łukasz DUTKA, Renata G. SŁOTA, Jacek KITOWSKI // W: Computational Science – ICCS 2021 : 21st international conference : Krakow, Poland, June 16–18, 2021 : proceedings, Pt. 1 / eds. Maciej Paszyński, [et al.]. — Cham : Springer Nature Switzerland, cop. 2021. — (Lecture Notes in Computer Science ; ISSN 0302-9743 ; LNCS 12742. Theoretical Computer Science and General Issues ; ISSN 0302-9743). — ISBN: 978-3-030-77960-3; e-ISBN: 978-3-030-77961-0. — S. 243–255. — Bibliogr., Abstr. — Publikacja dostępna online od: 2021-06-09. — J. Kitowski - dod. afiliacja: ACK Cyfronet AGH


Autorzy (5)


Słowa kluczowe

scientific workflow managementKuberneteshybrid cloudcloud burstingscientific data management

Dane bibliometryczne

ID BaDAP134700
Data dodania do BaDAP2021-06-23
DOI10.1007/978-3-030-77961-0_21
Rok publikacji2021
Typ publikacjimateriały konferencyjne (aut.)
Otwarty dostęptak
WydawcaSpringer
Konferencja21st International Conference on Computational Science
Czasopisma/serieLecture Notes in Computer Science, Theoretical Computer Science and General Issues

Abstract

Cloud bursting is an application deployment model wherein additional computing resources are provisioned from public clouds in cases where local resources are not sufficient, e.g. during peak demand periods. We propose and experimentally evaluate a cloud-bursting solution for scientific workflows. Our solution is portable thanks to using Kubernetes for deployment of the workflow management system and computing clusters in multiple clouds. We also introduce transparent data access by employing a virtual distributed file system across the clouds, allowing jobs to use a POSIX file system interface, while hiding data transfer between clouds. To balance load distribution and minimize the communication volume between clouds, we leverage graph partitioning, while ensuring that the algorithm distributes the load equally at each parallel execution stage of a workflow. The solution is experimentally evaluated using the HyperFlow workflow management system integrated with the Onedata data management platform, deployed in our on-premise cloud in Cyfronet AGH and in the Google Cloud.

Publikacje, które mogą Cię zainteresować

fragment książki
Transparent data access for scientific workflows across clouds / Michał ORZECHOWSKI, Bartosz BALIŚ, Łukasz DUTKA, Renata G. SŁOTA, Jacek KITOWSKI // W: Euro-Par 2019: parallel processing workshops : Euro-Par 2019 international workshops : Göttingen, Germany, August 26–30, 2019 : revised selected papers / eds. Ulrich Schwardmann, [et al.]. — Cham : Springer Nature Switzerland AG, cop. 2020. — (Lecture Notes in Computer Science ; ISSN 0302-9743 ; LNCS 11997. Theoretical Computer Science and General Issues ; ISSN 0302-9743). — ISBN: 978-3-030-48339-5; e-ISBN: 978-3-030-48340-1. — S. 751–755. — Bibliogr. s. 754–755, Abstr. — Publikacja dostępna online od: 2020-05-29. — J. Kitowski - dod. afiliacja: ACK Cyfronet AGH
fragment książki
Supermodeling - a meta-procedure for data assimilation and parameters estimation / Leszek SIWIK, Marcin ŁOŚ, Witold DZWINEL // W: Computational Science – ICCS 2021 : 21st international conference : Krakow, Poland, June 16–18, 2021 : proceedings, Pt. 2 / eds. Maciej Paszyński, [et al.]. — Cham : Springer Nature Switzerland, cop. 2021. — (Lecture Notes in Computer Science ; ISSN 0302-9743 ; LNCS 12743. Theoretical Computer Science and General Issues ; ISSN 0302-9743). — ISBN: 978-3-030-77963-4; e-ISBN: 978-3-030-77964-1. — S. 358–372. — Bibliogr., Abstr. — Publikacja dostępna online od: 2021-06-09