NobleBlocks

Research indexJournal article

Apache Spark: A Big Data Processing Engine

Shaikh, Mohiuddin, Alufaisan and 1 other

2019 2nd IEEE Middle East and North Africa COMMunications Conference (MENACOMM)1 Nov 2019

NobleIDark:/48914/w1/20260524/71578761DOI10.1109/menacomm46666.2019.8988541

Paywalled

The full text is at the publisher

You can read the abstract here, or open it at the publisher.

Read at the publisher

Abstract

Big data analysis has influenced the industry market. It has a significant impact on large and varied datasets to exhibit the hidden patterns and other revelations. Apache Hadoop, Apache Flink and Apache Storm are some commonly used frameworks for big data analysis. Apache Spark is a consolidated big data analytics engine and provides absolute data parallelism. This paper scrutinizes a technical review on big data analytics using Apache Spark and how it uses in-memory computation that makes it remarkably faster as compared to other corresponding frameworks. Moreover, Spark also provides exceptional batch processing and stream processing capabilities. Furthermore, it also discuses over the multithreading and concurrency capabilities of Apache Spark. The point of convergence is architecture, hardware requirements, ecosystem, use cases, features of Apache Spark and the use of Spark in emerging technologies.

Discussion

Citations

Topics

More

Suggest a correction or report