NobleBlocks

Research indexJournal article

Multi-Level Elasticity for Data Stream Processing

Marangozova‐Martin, De Palma, Rheddane

IEEE Transactions on Parallel and Distributed Systems27 Mar 2019

DOI10.1109/tpds.2019.2907950

Paywalled

The full text is at the publisher

You can read the abstract here, or open it at the publisher.

Read at the publisher

Abstract

This paper investigates reactive elasticity in stream processing environments where the performance goal is to analyze large amounts of data with low latency and minimum resources. Working in the context of Apache Storm, we propose an elastic management strategy which modulates the parallelism degree of applications' components while explicitly addressing the hierarchy of execution containers (virtual machines, processes and threads). We show that provisioning the wrong kind of container may lead to performance degradation and propose a solution that provisions the least expensive container (with minimum resources) to increase performance. We describe our monitoring metrics and show how we take into account the specifics of an execution environment. We provide an experimental evaluation with real-world applications which validates the applicability of our approach.

Discussion

Citations

Topics

More

Suggest a correction or report