• Title/Summary/Keyword: Bigdata utilization

Search Result 42, Processing Time 0.027 seconds

A Study of Big Data Domain Automatic Classification Using Machine Learning (머신러닝을 이용한 빅데이터 도메인 자동 판별에 관한 연구)

  • Kong, Seongwon;Hwang, Deokyoul
    • The Journal of Bigdata
    • /
    • v.3 no.2
    • /
    • pp.11-18
    • /
    • 2018
  • This study is a study on domain automatic classification for domain - based quality diagnosis which is a key element of big data quality diagnosis. With the increase of the value and utilization of Big Data and the rise of the Fourth Industrial Revolution, the world is making efforts to create new value by utilizing big data in various fields converged with IT such as law, medical, and finance. However, analysis based on low-reliability data results in critical problems in both the process and the result, and it is also difficult to believe that judgments based on the analysis results. Although the need of highly reliable data has also increased, research on the quality of data and its results have been insufficient. The purpose of this study is to shorten the work time to automizing the domain classification work which was performed from manually to using machine learning in the domain - based quality diagnosis, which is a key element of diagnostic evaluation for improving data quality. Extracts information about the characteristics of the data that is stored in the database and identifies the domain, and then featurize it, and automizes the domain classification using machine learning. We will use it for big data quality diagnosis and contribute to quality improvement.

Efficient distributed consensus optimization based on patterns and groups for federated learning (연합학습을 위한 패턴 및 그룹 기반 효율적인 분산 합의 최적화)

  • Kang, Seung Ju;Chun, Ji Young;Noh, Geontae;Jeong, Ik Rae
    • Journal of Internet Computing and Services
    • /
    • v.23 no.4
    • /
    • pp.73-85
    • /
    • 2022
  • In the era of the 4th industrial revolution, where automation and connectivity are maximized with artificial intelligence, the importance of data collection and utilization for model update is increasing. In order to create a model using artificial intelligence technology, it is usually necessary to gather data in one place so that it can be updated, but this can infringe users' privacy. In this paper, we introduce federated learning, a distributed machine learning method that can update models in cooperation without directly sharing distributed stored data, and introduce a study to optimize distributed consensus among participants without an existing server. In addition, we propose a pattern and group-based distributed consensus optimization algorithm that uses an algorithm for generating patterns and groups based on the Kirkman Triple System, and performs parallel updates and communication. This algorithm guarantees more privacy than the existing distributed consensus optimization algorithm and reduces the communication time until the model converges.

Research of Performance Interference Control Technique for Heterogeneous Services in Bigdata Platform (빅데이터 플랫폼에서 이종 서비스간 성능 간섭 현상 제어에 관한 연구)

  • Jin, Kisung;Lee, Sangmin;Kim, Youngkyun
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.6
    • /
    • pp.284-289
    • /
    • 2016
  • In the Hadoop-based Big Data analysis model, the data movement between the legacy system and the analysis system is difficult to avoid. To overcome this problem, a unified Big Data file system is introduced so that a unified platform can support the legacy service as well as the analysis service. However, major challenges in avoiding the performance degradation problem due to the interference of two services remain. In order to solve this problem, we first performed a real-life simulation and observed resource utilization, workload characteristics and I/O balanced level. Based on this analysis, two solutions were proposed both for the system level and for the technical level. In the system level, we divide I/O path into the legacy I/O path and the analysis I/O path. In the technical level, we introduce an aggressive prefetch method for analysis service which requires the sequential read. Also, we introduce experimental results that shows the outstanding performance gain comparing the previous system.

A study on the effect of SME IT resource on performance (중소기업의 IT자원이 업무성과에 미치는 영향에 관한 연구)

  • Jin, Jeongsuk;Park, Jooseok;Park, Jaehong
    • The Journal of Bigdata
    • /
    • v.4 no.2
    • /
    • pp.141-158
    • /
    • 2019
  • Based on RBV(Resource Based View), IT of SMEs classified into IT resource and capabilities. And We confirmed that capabilities and resources affected each performance. In other words, based on the questionnaire of SMEs and IT professionals, divides capability from the overall IT resource that are possessed by SMEs. Among the four attributes (value, rare, non-substitutability, imperfect imitability) presented by Barney (1991), this study targeted at value and imperfect imitability and investigated how SMEs recognize IT resource and capability. Furthermore, this study tests how IT resource and capability influence corporate performance. The result of this study finds that resources that are needed on "Knowledge-based" are classified into IT capability, otherwise classified into IT resource. Analysis shows that server, DB(database), system administrators, programmers, CIO, BA were capabilities, Desktop PC, PC software, software for salary and accounting management, e-commerce, Homepage, and network inside th enterprise were resources. Secondly, this study reveals that both IT resource and IT capability affected company performance (employee satisfaction, CEO satisfaction). IT is certainly having an impact on corporate performance. In conclusion, resource can be either IT resource or IT capability based on they way of utilization. And both IT resource and IT capability have an influence on corporate performance (employee job satisfaction, CEO satisfaction). Therefore, when considering IT investment, a company can purchase necessary IT resource and actively utilize it to be IT capability, which can have an influence on corporate performance in return.

  • PDF

Comparative Analysis for Clustering Based Optimal Vehicle Routes Planning (클러스터링 기반의 최적 차량 운행 계획 수립을 위한 비교연구)

  • Kim, Jae-Won;Shin, KwangSup
    • The Journal of Bigdata
    • /
    • v.5 no.1
    • /
    • pp.155-180
    • /
    • 2020
  • It takes the most important role the problem of assigining vehicles and desigining optimal routes for each vehicle in order to enhance the logistics service level. While solving the problem, various cost factors such as number of vehicles, the capacity of vehicles, total travelling distance, should be considered at the same time. Although most of logistics service providers introduced the Transportation Management System (TMS), the system has the limitation which can not consider the practical constraints. In order to make the solution of TMS applicable, it is required experts revised the solution of TMS based on their own experience and intuition. In this research, different from previous research which have focused on minimizing the total cost, it has been proposed the methodology which can enhance the efficiency and fairness of asset utilization, simultaneously. First of all, it has been adopted the Cluster-First Route-Second (CFRS) approach. Based on the location of customers, we have grouped customers as clusters by using four different clustering algorithm such as K-Means, K-Medoids, DBSCAN, Model-based clustering and a procedural approach, Fisher & Jaikumar algorithm. After getting the result of clustering, it has been developed the optiamal vehicle routes within clusters. Based on the result of numerical experiments, it can be said that the propsed approach based on CFRS may guarantee the better performance in terms of total travelling time and distance. At the same time, the variance of travelling distance and number of visiting customers among vehicles, it can be concluded that the proposed approach can guarantee the better performance of assigning tasks in terms of fairness.

A Study on Bigdata Utilization in Cultural and Artistic Contents Production and Distribution (문화예술 콘텐츠 제작 및 유통에서의 빅데이터 활용 연구)

  • Kim, Hyun-Young;Kim, Jae-Woong
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.7
    • /
    • pp.384-392
    • /
    • 2019
  • Big data-related research that deals with the amount of explosive information in the era of the Fourth Industrial Revolution is actively underway. Big data is an essential element that promotes the development of artificial intelligence with a wide range of data that become learning data for machine learning, or deep learning. The use of deep learning and big data in various fields has produced meaningful results. In this paper, we have investigated the use of Big Data in the cultural arts industry, focusing on video contents. Noteworthy is that big data is used not only in the distribution of cultural and artistic contents but also in the production stage. In particular, we first looked at what kind of achievements and changes the Netflix in the US brought to the OTT business, and analyzed the current state of the OTT business in Korea. After that, Netflix analyzed the success stories of 'House of Cards', which was produced / circulated through 'Deep Learning' cinematique, which is a prediction algorithm, through accumulated customer data. After that, FGI (Focus Group Interview) was held for cultural and artistic contents experts. In this way, the future prospects of Big Data in the domestic culture and arts industry are divided into technical aspect, creative aspect, and ethical aspect.

Digital Transformation Based on Chatbot in Legacy Environment (챗봇을 이용한 Legacy 환경의 Digital Transformation)

  • Jang, Jeong-ho;Kim, Jin-soo;Lee, Kang-Yoon
    • The Journal of Bigdata
    • /
    • v.3 no.2
    • /
    • pp.79-85
    • /
    • 2018
  • As the utilization of chatbots grows and the AI market grows, many companies are interested. And everybody is spurring growth by offering chatbot build services so that they can create chatbots. This makes chatbots easier to service on the messenger platform, which is changing the existing application market. In this paper, we present a methodology for designing and implementing existing DB-based applications as instant messenger platform-based applications, and summarize what to consider in actual implementation to provide an optimal system structure. According to this methodology, we design and implement a chatbot that serves as an teaching advisor who provides information to the students in the curriculum. The implemented application objectively visualizes the user's desired information from the user's point of view and delivers it through the interactive interface quickly and intuitively. By implementing these services and real service, it is predicted that DB-based information providing applications will be implemented as chatbots and will be changed to bi-directional communication through an interactive interface. it is predicted that DB-based information providing applications will be implemented as chatbots and will be changed to bi-directional communication through an interactive interface. Enterprise legacy application will take chatbot technology as one of important digital transformation initiative.

A Study on AI Industrial Ecosystem to Foster Artificial Intelligence Industry in Busan (부산지역 인공지능 산업 육성을 위한 AI 산업생태계 연구)

  • Bae, Soohyun;Kim, Sungshin;Jeong, Seok Chan
    • The Journal of Bigdata
    • /
    • v.5 no.2
    • /
    • pp.121-133
    • /
    • 2020
  • This study was carried out to set the direction of the new industry policy of Busan city by analyzing the changing trend of artificial intelligence technology that has recently developed rapidly and predicting the direction of future development. The company wanted to draw up support measures to utilize artificial intelligence technology, which has been rapidly emerging in the market, in the region's specialized industry. Artificial intelligence is a key keyword in the fourth industrial revolution and artificial intelligence-based data utilization technology can be used in various fields from manufacturing processes to services, and is entering an era of super-fusion in which barriers between technologies and industries will be broken down. In this study, the direction of promotion for fostering Busan as an artificial intelligence city was derived based on the comparison and analysis of artificial intelligence-related ecosystems among major local governments. In this study, we wanted to present a plan to create an artificial intelligence industrial ecosystem that can be called a key policy to foster Busan as an 'AI City'. Busan's plan to foster the AI industry ecosystem is aimed at establishing a policy direction to ultimately nurture the artificial intelligence industry as Busan's future food source.

Data-Driven Technology Portfolio Analysis for Commercialization of Public R&D Outcomes: Case Study of Big Data and Artificial Intelligence Fields (공공연구성과 실용화를 위한 데이터 기반의 기술 포트폴리오 분석: 빅데이터 및 인공지능 분야를 중심으로)

  • Eunji Jeon;Chae Won Lee;Jea-Tek Ryu
    • The Journal of Bigdata
    • /
    • v.6 no.2
    • /
    • pp.71-84
    • /
    • 2021
  • Since small and medium-sized enterprises fell short of the securement of technological competitiveness in the field of big data and artificial intelligence (AI) field-core technologies of the Fourth Industrial Revolution, it is important to strengthen the competitiveness of the overall industry through technology commercialization. In this study, we aimed to propose a priority related to technology transfer and commercialization for practical use of public research results. We utilized public research performance information, improving missing values of 6T classification by deep learning model with an ensemble method. Then, we conducted topic modeling to derive the converging fields of big data and AI. We classified the technology fields into four different segments in the technology portfolio based on technology activity and technology efficiency, estimating the potential of technology commercialization for those fields. We proposed a priority of technology commercialization for 10 detailed technology fields that require long-term investment. Through systematic analysis, active utilization of technology, and efficient technology transfer and commercialization can be promoted.

A Study on the Current Status and Application Strategies of the Smart Devices in the Library (도서관에서의 스마트 디바이스 활용 현황분석 및 서비스 적용방안)

  • Kim, Tae-Young;Park, Tae-Yeon;Yang, Dongmin;Oh, Hyo-Jung
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.51 no.4
    • /
    • pp.203-226
    • /
    • 2017
  • The advent of the fourth industrial revolution has led to various technologies such as bigdata, the internet of things, artificial intelligence etc. Based on these innovations, the types of information services can changed in the library. The focus is on smart device. This study aims to identify utilization status and service implications of the smart device in the library. To achieve this goal, we conducted current status analysis of the smart device in the library through literature research and online search and gathered the executives views of practical librarians. Consequently, we proposed improvement of library service by using smart device. The results of this study will be expected to help next generation library establish service strategies.