International journal of advanced smart convergence

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Fabricator based on B+Tree for Metadata Management in Distributed Environment

  • Chae-Yeon Yun (Graduate School of Smart Convergence, KwangWoon University) ;
  • Seok-Jae Moon (Graduate School of Smart Convergence, KwangWoon University)
  • Received : 2024.07.16
  • Accepted : 2024.07.27
  • Published : 2024.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

In a distributed environment, data fabric refers to the technology and architecture that provides data management, integration, and access in a consistent and unified manner. To build a data fabric, it is necessary to maintain data consistency, establish a data governance system, reduce structural differences between data sources, and provide a unified view. In this paper, we propose the Fabricator system, a technology that provides data management and access in a consistent and unified manner by building a metadata registry. Fabricator manages the addition and modification of metadata schemas and matching processes by designing a matching tool called MetaSB Manager that applies B+Tree. This allows real-time integration of various data sources in a distributed environment, maximizing the flexibility and usability of data.

Keywords

Acknowledgement

This paper was supported by the KwangWoon University Research Grant of 2024.

References

  1. Patel and N. C. Debnath, Data Science with Semantic Technologies. CRC Press, pp. 267-286, 2023. DOI: https://doi.org/10.1007/978-3-658-12225-6_4
  2. Underwood, "Continuous Metadata in Continuous Integration, Stream Processing and Enterprise DataOps," Data Intelligence, vol. 5, no. 1. MIT Press, pp. 275-288, 2023. DOI: https://doi.org/10.1162/dint_a_00193
  3. Li, M. Yang, X. Xia, K. Zhang, and K. Liu, "A Distributed Data Fabric Architecture based on Metadate Knowledge Graph," 2022 5th International Conference on Data Science and Information Technology (DSIT). IEEE, Jul. 22, 2022. DOI: https://doi.org/10.1109/DSIT55514.2022.9943831
  4. Liu, M. Yang, X. Li, K. Zhang, X. Xia, and H. Yan, "M-Data-Fabric: A Data Fabric System Based on Metadata," 2022 IEEE 5th International Conference on Big Data and Artificial Intelligence (BDAI). IEEE, Jul. 08, 2022. DOI: https://doi.org/10.1109/BDAI56143.2022.9862807
  5. V. Sharma, B. Balusamy, J. J. Thomas, and L. G. Atlas, Eds., "Data Fabric Architectures." De Gruyter, May 04, 2023. DOI: https://doi.org/10.1515/9783111000886
  6. C.-Y. Yun and S.-J. Moon, "A Fabricator Design for Metadata CI/CD in Data Fabric," International Journal of Internet, Broadcasting and Communication, vol. 16, no. 2, pp. 193-202, May 2024. DOI: https://doi.org/10.7236/IJIBC.2024.16.2.193
  7. F. Nawab and M. Sadoghi, "Consensus in Data Management: From Distributed Commit to Blockchain," Foundations and Trends(R) in Databases, vol. 12, no. 4. Now Publishers, pp. 221-364, 2023. DOI: https://doi.org/10.1561/1900000075
  8. J. Liang, D. Hu, and J. Feng, "Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation." arXiv, 2020. DOI: https ://doi.org/10.48550/arXiv.2002.08546
  9. Kim, Seon Hwan and Kwak, Jong Wook, "Garbage Collection Method using Proxy Block considering Index Data Structure based on Flash Memory," Journal of the Korea Society of Computer and Information, vol. 20, no. 6, pp. 1-11, Jun. 2015. DOI: https://doi.org/10.9708/JKSCI.2015.20.6.001
  10. P. Chalermsook, M. Goswami, L. Kozma, K. Mehlhorn, and T. Saranurak, "Multi-finger binary search trees." arXiv, 2018. DOI: https://doi.org/10.48550/ARXIV.1809.01759