Szczegóły publikacji
Opis bibliograficzny
Scientific workflow management on hybrid clouds with cloud bursting and transparent data access / Bartosz BALIŚ, Michał ORZECHOWSKI, Łukasz DUTKA, Renata G. SŁOTA, Jacek KITOWSKI // W: Computational Science – ICCS 2021 : 21st international conference : Krakow, Poland, June 16–18, 2021 : proceedings, Pt. 1 / eds. Maciej Paszyński, [et al.]. — Cham : Springer Nature Switzerland, cop. 2021. — (Lecture Notes in Computer Science ; ISSN 0302-9743 ; LNCS 12742. Theoretical Computer Science and General Issues ; ISSN 0302-9743). — ISBN: 978-3-030-77960-3; e-ISBN: 978-3-030-77961-0. — S. 243–255. — Bibliogr., Abstr. — Publikacja dostępna online od: 2021-06-09. — J. Kitowski - dod. afiliacja: ACK Cyfronet AGH
Autorzy (5)
Słowa kluczowe
Dane bibliometryczne
| ID BaDAP | 134700 |
|---|---|
| Data dodania do BaDAP | 2021-06-23 |
| DOI | 10.1007/978-3-030-77961-0_21 |
| Rok publikacji | 2021 |
| Typ publikacji | materiały konferencyjne (aut.) |
| Otwarty dostęp |  |
| Wydawca | Springer |
| Konferencja | 21st International Conference on Computational Science |
| Czasopisma/serie | Lecture Notes in Computer Science, Theoretical Computer Science and General Issues |
Abstract
Cloud bursting is an application deployment model wherein additional computing resources are provisioned from public clouds in cases where local resources are not sufficient, e.g. during peak demand periods. We propose and experimentally evaluate a cloud-bursting solution for scientific workflows. Our solution is portable thanks to using Kubernetes for deployment of the workflow management system and computing clusters in multiple clouds. We also introduce transparent data access by employing a virtual distributed file system across the clouds, allowing jobs to use a POSIX file system interface, while hiding data transfer between clouds. To balance load distribution and minimize the communication volume between clouds, we leverage graph partitioning, while ensuring that the algorithm distributes the load equally at each parallel execution stage of a workflow. The solution is experimentally evaluated using the HyperFlow workflow management system integrated with the Onedata data management platform, deployed in our on-premise cloud in Cyfronet AGH and in the Google Cloud.