NobleBlocks

Research indexJournal article

DATA-DRIVEN ACCURACY ESTIMATION IN BIG DATA PROCESSING THROUGH MACHINE LEARNING

Chunduru

INTERNATIONAL JOURNAL OF COMPUTER ENGINEERING & TECHNOLOGY30 Oct 2024Open access

NobleIDark:/48914/w1/20261105/F2C6D61BDOI10.34218/ijcet_15_05_111

Free to read at the publisher

You can read the abstract here, or the full text at the publisher.

Read at the publisher

Abstract

Big data has emerged as a transformative force in modern decision-making and technological innovation, characterized by massive amounts of complex data generated from diverse sources, including social media, sensors, healthcare systems, and industrial equipment.This study investigates the performance characteristics of big data systems through the analysis of processing metrics across various data sizes.The research uses advanced machine learning techniques, specifically XG Boost Regression and Gradient Boosting Regression, to predict analytical accuracy based on key parameters including data size, processing speed, node utilization, and query load.The dataset consists of 20 observations with data sizes ranging from 9.8 to 20.3 TB, demonstrating strong positive correlations between all variables (0.98-1.00).XG Boost Regression achieved 0.7804 R² on the test data, while Gradient Boosting Regression showed superior performance with 0.9278 R², indicating better predictive ability and generalization.The analysis reveals that the system maintains high utilization rates ranging from 88% to 96%, suggesting well-optimized resource allocation and effective load balancing.Both models successfully captured the complex relationships between processing parameters and analytical accuracy, confirming their effectiveness for big data performance modeling and prediction in enterprise environments.

Discussion

More

Suggest a correction or report