• Title/Summary/Keyword: Big-Data Platform

Search Result 506, Processing Time 0.032 seconds

A Study on the Development of Phased Big Data Distribution Model Based on Big Data Distribution Ecology (빅데이터 유통 생태계에 기반한 단계별 빅데이터 유통 모델 개발에 관한 연구)

  • Kim, Shinkon;Lee, Sukjun;Kim, Jeonggon
    • Journal of Digital Convergence
    • /
    • v.14 no.5
    • /
    • pp.95-106
    • /
    • 2016
  • The major thrust of this research focuses on the development of phased big data distribution model based on the big data ecosystem. This model consists of 3 phases. In phase 1, data intermediaries are participated in this model and transaction functions are provided. This system consists of general control systems, registrations, and transaction management systems. In phase 2, trading support systems with data storage, analysis, supply, and customer relation management functions are designed. In phase 3, transaction support systems and linked big data distribution portal systems are developed. Recently, emerging new data distribution models and systems are evolving and substituting for past data management system using new technology and the processes in data science. The proposed model may be referred as criteria for industrial standard establishment for big data distribution and transaction models in the future.

Design of a Large-scale Task Dispatching & Processing System based on Hadoop (하둡 기반 대규모 작업 배치 및 처리 기술 설계)

  • Kim, Jik-Soo;Cao, Nguyen;Kim, Seoyoung;Hwang, Soonwook
    • Journal of KIISE
    • /
    • v.43 no.6
    • /
    • pp.613-620
    • /
    • 2016
  • This paper presents a MOHA(Many-Task Computing on Hadoop) framework which aims to effectively apply the Many-Task Computing(MTC) technologies originally developed for high-performance processing of many tasks, to the existing Big Data processing platform Hadoop. We present basic concepts, motivation, preliminary results of PoC based on distributed message queue, and future research directions of MOHA. MTC applications may have relatively low I/O requirements per task. However, a very large number of tasks should be efficiently processed with potentially heavy inter-communications based on files. Therefore, MTC applications can show another pattern of data-intensive workloads compared to existing Hadoop applications, typically based on relatively large data block sizes. Through an effective convergence of MTC and Big Data technologies, we can introduce a new MOHA framework which can support the large-scale scientific applications along with the Hadoop ecosystem, which is evolving into a multi-application platform.

A Study on the Accumulation and Use of Corporate Records: Corporate Records Management as a Big Data Platform (기업의 현용기록 축적과 이용 방안 연구: 빅데이터 플랫폼으로서의 기업기록관리)

  • Kim, Sung-woo;Rieh, Hae-young
    • Journal of Korean Society of Archives and Records Management
    • /
    • v.20 no.3
    • /
    • pp.99-118
    • /
    • 2020
  • The creation of value and the enhancement of benefits through records management by enterprises are comparable to those by public institutions. However, Korea has yet to establish guidelines on corporate records management. Global companies are strengthening their competitiveness by reducing trial and error in their work through the accumulation and use of records as information assets, which serve as the output of their work processes. While Korean companies agree on the necessity of corporate records management, most of them are concerned with archival (noncurrent records) management, such as historical compilation and historical data management, rather than records (current record) management. Therefore, through a case study of a K-company with effective records management, this study identifies methods to promote the accumulation, use, and management of corporate records in line with the search of value and benefits. Moreover, the company emphasizes the management of corporate records as a big data platform that accumulates and uses data, which is an important resource in the era of the Fourth Industrial Revolution, and proposes measures for their revitalization.

Usefulness of RHadoop in Case of Healthcare Big Data Analysis (RHadoop을 이용한 보건의료 빅데이터 분석의 유효성)

  • Ryu, Wooseok
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2017.10a
    • /
    • pp.115-117
    • /
    • 2017
  • R has become a popular analytics platform as it provides powerful analytic functions as well as visualizations. However, it has a weakness in which scalability is limited. As an alternative, the RHadoop package facilitates distributed processing of R programs under the Hadoop platform. This paper investigates usefulness of the RHadoop package when analyzing healthcare big data that is widely open in the internet space. To do this, this paper has compared analytic performances of R and RHadoop using the medical treatment records of year 2015 provided by National Health Insurance Service. The result shows that RHadoop effectively enhances processing performance of healthcare big data compared with R.

  • PDF

A Study on the Development of E-Commerce Shipping Platform in China

  • Ying, Lou;Lee, Su-Ho;Shou, Jian-Min
    • Journal of Navigation and Port Research
    • /
    • v.40 no.2
    • /
    • pp.73-82
    • /
    • 2016
  • With the advent of Internet era, e-commerce has become the focus of human's life. It leads a new direction of social development. As the representative of traditional industry, shipping industry is confronted with a series of difficulties, which have to break through the traditional and existing model to make their business for survival. With the increasing pricking up of market competition, shipping industry is now facing development bottle neck, but e-commerce provides a new way to solve the problem. This paper firstly describes the existing forms of the e-commerce shipping platform. Secondly analyzes the data for the situation of shipping industry in China, the data for expected functions of an e-commerce shipping platform and the data for how to choose a specific e-commerce shipping platform. Thirdly analyzes the potential risks of establishing e-commerce shipping platform in China. Based on the above researched, the paper provides a suggested model of the shipping e-commerce shipping platform in China.

Leveraging Big Data for Spark Deep Learning to Predict Rating

  • Mishra, Monika;Kang, Mingoo;Woo, Jongwook
    • Journal of Internet Computing and Services
    • /
    • v.21 no.6
    • /
    • pp.33-39
    • /
    • 2020
  • The paper is to build recommendation systems leveraging Deep Learning and Big Data platform, Spark to predict item ratings of the Amazon e-commerce site. Recommendation system in e-commerce has become extremely popular in recent years and it is very important for both customers and sellers in daily life. It means providing the users with products and services they are interested in. Therecommendation systems need users' previous shopping activities and digital footprints to make best recommendation purpose for next item shopping. We developed the recommendation models in Amazon AWS Cloud services to predict the users' ratings for the items with the massive data set of Amazon customer reviews. We also present Big Data architecture to afford the large scale data set for storing and computation. And, we adopted deep learning for machine learning community as it is known that it has higher accuracy for the massive data set. In the end, a comparative conclusion in terms of the accuracy as well as the performance is illustrated with the Deep Learning architecture with Spark ML and the traditional Big Data architecture, Spark ML alone.

A study on the analysis of virtual reality platform API for virtual reality (VR) development

  • Lee, Byong-Kwon
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.8
    • /
    • pp.23-30
    • /
    • 2020
  • As the 4th industrial revolution emerged, the latest technologies such as IoT, AI, Big data, AR/VR/XR are emerging. However, in the field of virtual reality (VR) technology platform services, there is no standardization and systematic support. In addition, various platform technologies related to virtual reality have been presented, making it difficult to select an API that should be selected for development. In this study, we analyzed the method for virtual reality development and the virtual reality (VR) technology that is being serviced by users. In addition, by presenting the advantages and disadvantages of each development platform, we intend to present a reference point for developers to select an efficient platform. In addition, it will help the developer to select an effective equipment and software platform in comparison with the advantages and disadvantages of various HMD devices used in virtual reality. The virtual reality (VR) development environment test used products from Oculus, and the software development environment was tested with two types: WebBased VR and HMD embedded.

Analysis of Meta Fashion Meaning Structure using Big Data: Focusing on the keywords 'Metaverse' + 'Fashion design' (빅데이터를 활용한 메타패션 의미구조 분석에 관한 연구: '메타버스' + '패션디자인' 키워드를 중심으로)

  • Ji-Yeon Kim;Shin-Young Lee
    • Fashion & Textile Research Journal
    • /
    • v.25 no.5
    • /
    • pp.549-559
    • /
    • 2023
  • Along with the transition to the fourth industrial revolution, the possibility of metaverse-based innovation in the fashion field has been confirmed, and various applications are being sought. Therefore, this study performs meaning structure analysis and discusses the prospects of meta fashion using big data. From 2020 to 2022, data including the keyword "metaverse + fashion design" were collected from portal sites (Naver, Daum, and Google), and the results of keyword frequency, N-gram, and TF-IDF analyses were derived using text mining. Furthermore, network visualization and CONCOR analysis were performed using Ucinet 6 to understand the interconnected structure between keywords and their essential meanings. The results were as follows: The main keywords appeared in the following order: fashion, metaverse, design, 3D, platform, apparel, and virtual. In the N-gram analysis, the density between fashion and metaverse words was high, and in the TF-IDF analysis results, the importance of content- and technology-related words such as 3D, apparel, platform, NFT, education, AI, avatar, MCM, and meta-fashion was confirmed. Through network visualization and CONCOR analysis using Ucinet 6, three cluster results were derived from the top emerging words: "metaverse fashion design and industry," "metaverse fashion design and education," and "metaverse fashion design platform." CONCOR analysis was also used to derive differentiated analysis results for middle and lower words. The results of this study provide useful information to strengthen competitiveness in the field of metaverse fashion design.

A Big-Data Trajectory Combination Method for Navigations using Collected Trajectory Data (수집된 경로데이터를 사용하는 내비게이션을 위한 대용량 경로조합 방법)

  • Koo, Kwang Min;Lee, Taeho;Park, Heemin
    • Journal of Korea Multimedia Society
    • /
    • v.19 no.2
    • /
    • pp.386-395
    • /
    • 2016
  • In trajectory-based navigation systems, a huge amount of trajectory data is needed for efficient route explorations. However, it would be very hard to collect trajectories from all the possible start and destination combinations. To provide a practical solution to this problem, we suggest a method combining collected GPS trajectories data into additional generated trajectories with new start and destination combinations without road information. We present a trajectory combination algorithm and its implementation with Scala programming language on Spark platform for big data processing. The experimental results proved that the proposed method can effectively populate the collected trajectories into valid trajectory paths more than three hundred times.

Predictive Analysis of Financial Fraud Detection using Azure and Spark ML

  • Priyanka Purushu;Niklas Melcher;Bhagyashree Bhagwat;Jongwook Woo
    • Asia pacific journal of information systems
    • /
    • v.28 no.4
    • /
    • pp.308-319
    • /
    • 2018
  • This paper aims at providing valuable insights on Financial Fraud Detection on a mobile money transactional activity. We have predicted and classified the transaction as normal or fraud with a small sample and massive data set using Azure and Spark ML, which are traditional systems and Big Data respectively. Experimenting with sample dataset in Azure, we found that the Decision Forest model is the most accurate to proceed in terms of the recall value. For the massive data set using Spark ML, it is found that the Random Forest classifier algorithm of the classification model proves to be the best algorithm. It is presented that the Spark cluster gets much faster to build and evaluate models as adding more servers to the cluster with the same accuracy, which proves that the large scale data set can be predictable using Big Data platform. Finally, we reached a recall score with 0.73, which implies a satisfying prediction quality in predicting fraudulent transactions.