Research indexJournal article
Apache Spark: A Big Data Processing Engine
Shaikh, Mohiuddin, Alufaisan and 1 other
2019 2nd IEEE Middle East and North Africa COMMunications Conference (MENACOMM)1 Nov 2019
NobleIDark:/48914/w1/20260524/71578761DOI10.1109/menacomm46666.2019.8988541
Apache Spark: A Big Data Processing Engine
Shaikh, Mohiuddin, Alufaisan and 1 other · 2019 2nd IEEE Middle East and North Africa COMMunications Conference (MENACOMM) · 2019 · Subscription
NobleIDark:/48914/w1/20260524/71578761DOI10.1109/menacomm46666.2019.8988541
The full text is at the publisher
You can read the abstract here, or open it at the publisher.
Abstract
Big data analysis has influenced the industry market. It has a significant impact on large and varied datasets to exhibit the hidden patterns and other revelations. Apache Hadoop, Apache Flink and Apache Storm are some commonly used frameworks for big data analysis. Apache Spark is a consolidated big data analytics engine and provides absolute data parallelism. This paper scrutinizes a technical review on big data analytics using Apache Spark and how it uses in-memory computation that makes it remarkably faster as compared to other corresponding frameworks. Moreover, Spark also provides exceptional batch processing and stream processing capabilities. Furthermore, it also discuses over the multithreading and concurrency capabilities of Apache Spark. The point of convergence is architecture, hardware requirements, ecosystem, use cases, features of Apache Spark and the use of Spark in emerging technologies.