Szczegóły publikacji
Opis bibliograficzny
Storage-aware algorithms for scheduling of workflow ensembles in clouds / Piotr Bryk, Maciej MALAWSKI, Gideon Juve, Ewa Deelman // Journal of Grid Computing ; ISSN 1570-7873. — 2016 — vol. 14 iss. 2, s. 359–378. — Bibliogr. s. 377–378, Abstr. — Publikacja dostępna online: 2015-11-17
Autorzy (4)
- Bryk Piotr
- AGHMalawski Maciej
- Juve Gideon
- Deelman Ewa
Słowa kluczowe
Dane bibliometryczne
| ID BaDAP | 98107 |
|---|---|
| Data dodania do BaDAP | 2016-06-10 |
| Tekst źródłowy | URL |
| DOI | 10.1007/s10723-015-9355-6 |
| Rok publikacji | 2016 |
| Typ publikacji | artykuł w czasopiśmie |
| Otwarty dostęp |  |
| Creative Commons | |
| Czasopismo/seria | Journal of Grid Computing |
Abstract
This paper focuses on data-intensive workflows and addresses the problem of scheduling workflow ensembles under cost and deadline constraints in Infrastructure as a Service (IaaS) clouds. Previous research in this area ignores file transfers between workflow tasks, which, as we show, often have a large impact on workflow ensemble execution. In this paper we propose and implement a simulation model for handling file transfers between tasks, featuring the ability to dynamically calculate bandwidth and supporting a configurable number of replicas, thus allowing us to simulate various levels of congestion. The resulting model is capable of representing a wide range of storage systems available on clouds: from in-memory caches (such as memcached), to distributed file systems (such as NFS servers) and cloud storage (such as Amazon S3 or Google Cloud Storage). We observe that file transfers may have a significant impact on ensemble execution; for some applications up to 90 % of the execution time is spent on file transfers. Next, we propose and evaluate a novel scheduling algorithm that minimizes the number of transfers by taking advantage of data caching and file locality. We find that for data-intensive applications it performs better than other scheduling algorithms. Additionally, we modify the original scheduling algorithms to effectively operate in environments where file transfers take non-zero time.
