Toggle Main Menu Toggle Search

Open Access padlockePrints

Cost efficient scheduling of MapReduce applications on public clouds

Lookup NU author(s): Professor Raj Ranjan

Downloads

Full text for this publication is not currently held within this repository. Alternative links are provided below where available.


Abstract

© 2017 Elsevier B.V. MapReduce framework has been one of the most prominent ways for efficient processing large amount of data requiring huge computational capacity. On-demand computing resources of Public Clouds have become a natural host for these MapReduce applications. However, the decision of what type and in what amount computing and storage resources should be rented is still a user's responsibility. This is not a trivial task particularly when users may have performance constraints such as deadline and have several Cloud product types to choose with the intention of not spending much money. Even though there are several existing scheduling systems, however, most of them are not developed to manage the scheduling of MapReduce applications. That is, they do not consider things such as number of map and reduce tasks that are needed to be scheduled and heterogeneity of Virtual Machines (VMs) available. This paper proposes a novel greedy-based MapReduce application scheduling algorithm (MASA) that considers the user's constraints in order to minimize cost of renting Cloud resources while considering Service Level Agreements (SLA) in terms of the user given budget and deadline constraints. The simulation results show that MASA can achieve 25-50% cost reduction in comparison to current SLA agnostic methods and there is only 10% performance disparity between MASA and an exhaustive search algorithm.


Publication metadata

Author(s): Zeng X, Garg SK, Wen Z, Strazdins P, Zomaya AY, Ranjan R

Publication type: Article

Publication status: Published

Journal: Journal of Computational Science

Year: 2018

Volume: 26

Pages: 375-388

Print publication date: 01/05/2018

Online publication date: 09/08/2017

Acceptance date: 25/07/2017

ISSN (print): 1877-7503

ISSN (electronic): 1877-7511

Publisher: Elsevier BV

URL: https://doi.org/10.1016/j.jocs.2017.07.017

DOI: 10.1016/j.jocs.2017.07.017


Altmetrics

Altmetrics provided by Altmetric


Actions

Find at Newcastle University icon    Link to this publication


Share