Browse > Article
http://dx.doi.org/10.30693/SMJ.2018.7.1.9

Draft Design of DataLake Framework based on Abyss Storage Cluster  

Cha, ByungRae (광주과학기술원 전지전자컴퓨터공학부)
Park, Sun (광주과학기술원 전지전자컴퓨터공학부)
Shin, Byeong-Chun (전남대학교 수학과)
Kim, JongWon (광주과학기술원 전지전자컴퓨터공학부)
Publication Information
Smart Media Journal / v.7, no.1, 2018 , pp. 9-15 More about this Journal
Abstract
As an organization or organization grows in size, many different types of data are being generated in different systems. There is a need for a way to improve efficiency by processing data smarter in different systems. Just like DataLake, we are creating a single domain model that accurately describes the data and can represent the most important data for the entire business. In order to realize the benefits of a DataLake, it is import to know how a DataLake may be expected to work and what components architecturally may help to build a fully functional DataLake. DataLake components have a life cycle according to the data flow. And while th data flows into a DataLake from the point of acquisition, its meta-data is captured and managed along with data traceability, data lineage, and security aspects based on data sensitivity across its life cycle. According to this reason, we have designed the DataLake Framework based on Abyss Storage Cluster.
Keywords
DataLake Framework; Abyss Storage Cluster;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Tomcy John and Pankaj Misra, "Data Lake for Enterprises - Leveraging Lambda Architecture for Building Enterprise Data Lake," Packt Publishing, May 2017.
2 IBM의 빅데이터 정의, http://www.ibmbigdatahub.com/infographic/four-vs-big-data
3 장동인, "빅데이터로 일하는 기술," 한빛미디어, 2014년 12월 16일.
4 Mike barlow, "Real-Time Big Data Analytics: Emerging Architecture," 1st Edition, O'Reilly, Feb. 2013.
5 Pradeep Pasupuleti, Beulah Salome Purra, "Data Lake Development with Big Data," PACKT Publishing, 2015.
6 John Mallory and Robbie Wright, "Building Big Data Storage Solutions (Data Lakes) for Maximum Flexibility," Amazon Web Service, July 2017.
7 AWS, http://docs.aws.amazon.com/solutions/latest/data-lake- solution/architecture.html
8 AWS, https://aws.amazon.com/ko/big-data/datalake-on-aws/
9 차윤석 외 4인, "Abyss Storage의 Disk 타입에 의한 Ceph RADOS의 Benchmarking," 2017 한국통신학회 동계학술대회.
10 차병래 외 4인, "대용량 Abyss Storage의 KOREN 네트워크 기반 국내 및 해외 실증 테스트," 스마트미디어학회저널 Vol.6, no.1, pp.9-15, 2017년 3월호.
11 Lambda Architecture, http://searchbusinessanalytics.techtarget.com/definition/Lambda-architecture
12 차병래 외 4인, "Idea Sketch to Improvement Image Learning based on Machine Learning using Topology Theory," SMA 2017.
13 Cloud Bursting, http://searchcloudcomputing.techtarget.com/definition/cloud-bursting
14 Cloud Spanning, http://searchcloudcomputing.techtarget.com/definition/cloud-spanning
15 R, https://www.r-project.org/