A SYSTEMATIC REVIEW OF BIG DATA INTEGRATION CHALLENGES AND SOLUTIONS FOR HETEROGENEOUS DATA SOURCES

Abstract

This systematic review explores the current challenges and emerging solutions in big data integration, focusing on key issues such as semantic heterogeneity, data quality, scalability, and security. Using the PRISMA guidelines, 150 peer-reviewed articles were analyzed to identify both established and innovative approaches to integrating data from heterogeneous sources. The findings reveal that ontology-based frameworks are widely used to address semantic inconsistencies but face limitations in scalability when handling large, dynamic datasets. Machine learning has emerged as a powerful tool for automating data quality and schema matching processes, although its effectiveness is highly dependent on the availability of high-quality training data. Distributed computing frameworks like Hadoop and Spark have become the industry standard for scalable data integration, yet their implementation requires significant infrastructure and technical expertise. Cloud-based platforms offer flexible, scalable solutions, but concerns about data privacy and security persist. Blockchain technology, while promising for secure and decentralized data integration, is still in its infancy and struggles with scalability. The review highlights significant progress in the field but underscores the need for further research to address unresolved challenges in real-time integration, cross-domain data harmonization, and the management of unstructured data.