Toggle Main Menu Toggle Search

Open Access padlockePrints

Flower: A data analytics flow elasticity manager

Lookup NU author(s): Professor Raj Ranjan

Downloads


Licence

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND).


Abstract

© 2017 VLDB. A data analytics flow typically operates on three layers: ingestion, analytics, and storage, each of which is provided by a data-intensive system. These systems are often available as cloud managed services, enabling the users to have painfree deployment of data analytics flow applications such as click-stream analytics. Despite straightforward orchestration, elasticity management of the flows is challenging. This is due to: a) heterogeneity of workloads and diversity of cloud resources such as queue partitions, compute servers and NoSQL throughputs capacity, b) workload dependencies between the layers, and c) different performance behaviours and resource consumption patterns. In this demonstration, we present Flower, a holistic elasticity management system that exploits advanced optimization and control theory techniques to manage elasticity of complex data analytics flows on clouds. Flower analyzes statistics and data collected from different data-intensive systems to provide the user with a suite of rich functionalities, including: workload dependency analysis, optimal resource share analysis, dynamic resource provisioning, and cross-platform monitoring. We will showcase various features of Flower using a real-world data analytics flow. We will allow the audience to explore Flower by visually defining and configuring a data analytics flow elasticity manager and get hands-on experience with integrated data analytics flow management.


Publication metadata

Author(s): Khoshkbarforoushha A, Ranjan R, Wang Q, Friedrich C

Publication type: Conference Proceedings (inc. Abstract)

Publication status: Published

Conference Name: Proceedings of the 43rd International Conference on Very Large Databases

Year of Conference: 2017

Pages: 1893-1896

Online publication date: 01/08/2017

Acceptance date: 02/04/2016

Date deposited: 20/12/2017

ISSN: 2150-8097

Publisher: ACM

URL: http://www.vldb.org/pvldb/vol10/p1893-khoshkbarforoushha.pdf


Actions

Find at Newcastle University icon    Link to this publication


Share