• Title/Summary/Keyword: Big data campus

Search Result 51, Processing Time 0.023 seconds

Design of GlusterFS Based Big Data Distributed Processing System in Smart Factory (스마트 팩토리 환경에서의 GlusterFS 기반 빅데이터 분산 처리 시스템 설계)

  • Lee, Hyeop-Geon;Kim, Young-Woon;Kim, Ki-Young;Choi, Jong-Seok
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.11 no.1
    • /
    • pp.70-75
    • /
    • 2018
  • Smart Factory is an intelligent factory that can enhance productivity, quality, customer satisfaction, etc. by applying information and communications technology to the entire production process including design & development, manufacture, and distribution & logistics. The precise amount of data generated in a smart factory varies depending on the factory's size and state of facilities. Regardless, it would be difficult to apply traditional production management systems to a smart factory environment, as it generates vast amounts of data. For this reason, the need for a distributed big-data processing system has risen, which can process a large amount of data. Therefore, this article has designed a Gluster File System (GlusterFS)-based distributed big-data processing system that can be used in a smart factory environment. Compared to existing distributed processing systems, the proposed distributed big-data processing system reduces the system load and the risk of data loss through the distribution and management of network traffic.

The smart EV charging system based on the big data analysis of the power consumption patterns

  • Kang, Hun-Cheol;Kang, Ki-Beom;Ahn, Hyun-kwon;Lee, Seong-Hyun;Ahn, Tae-Hyo;Jwa, Jeong-Woo
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.9 no.2
    • /
    • pp.1-10
    • /
    • 2017
  • The high costs of electric vehicle supply equipment (EVSE) and installation are currently a stumbling block to the proliferation of electric vehicles (EVs). The cost-effective solutions are needed to support the expansion of charging infrastructure. In this paper, we develope EV charging system based on the big data analysis of the power consumption patterns. The developed EV charging system is consisted of the smart EV outlet, gateways, powergates, the big data management system, and mobile applications. The smart EV outlet is designed to low costs of equipment and installation by replacing the existing 220V outlet. We can connect the smart EV outlet to household appliances. Z-wave technology is used in the smart EV outlet to provide the EV power usage to users using Apps. The smart EV outlet provides 220V EV charging and therefore, we can restore vehicle driving range during overnight and work hours.

A Study on the Development of University Students Dropout Prediction Model Using Ensemble Technique (앙상블 기법을 활용한 대학생 중도탈락 예측 모형 개발)

  • Park, Sangsung
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.17 no.1
    • /
    • pp.109-115
    • /
    • 2021
  • The number of freshmen at universities is decreasing due to the recent decline in the school-age population, and the survival of many universities is threatened. To overcome this situation, universities are seeking ways to use big data within the school to improve the quality of education. A study on the prediction of dropout students is a representative case of using big data in universities. The dropout prediction can prepare a systematic management plan by identifying students who will drop out of school due to reasons such as dropout or expulsion. In the case of actual on-campus data, a large number of missing values are included because it is collected and managed by various departments. For this reason, it is necessary to construct a model by effectively reflecting the missing values. In this study, we propose a university student dropout prediction model based on eXtreme Gradient Boost that can be applied to data with many missing values and shows high performance. In order to examine the practical applicability of the proposed model, an experiment was performed using data from C University in Chungbuk. As a result of the experiment, the prediction performance of the proposed model was found to be excellent. The management strategy of dropout students can be established through the prediction results of the model proposed in this paper.

An Introduction and Trend Analysis in Questions of Engineer Big Data Analyst (빅데이터분석 기사 국가기술자격 개요 및 출제 경향 분석)

  • Jang, Hee-Seon;Song, Ji Young
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2022.01a
    • /
    • pp.393-394
    • /
    • 2022
  • 본 논문에서는 과학기술정보통신부와 통계청에서 주관하고 한국산업인력공단에서 시행(한국데이터산업진흥원 위탁)하는 「빅데이터분석기사」에 대한 필기 및 실기 시험의 내용을 설명하고 지금까지 2회에 걸쳐 시행된 시험에 대한 문제점과 이에 대한 해결방안을 제시하였다. 2021년 처음 시행된 국가기술자격으로써 기존 자격증과의 차별성, 난이도 조정, 수험생들의 각종 민원 발생 등의 문제를 해결하기 위한 체계적인 시스템 마련이 요구되며, 향후 데이터 과학자들에 대한 수요 급증에 대비하기 위해 빅데이터분석 실무 능력을 평가하기 위한 바람직한 제도와 정책이 병행되어야 한다.

  • PDF

Study of Efficient Algorithm for Deduplication of Complex Structure (복잡한 구조의 데이터 중복제거를 위한 효율적인 알고리즘 연구)

  • Lee, Hyeopgeon;Kim, Young-Woon;Kim, Ki-Young
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.14 no.1
    • /
    • pp.29-36
    • /
    • 2021
  • The amount of data generated has been growing exponentially, and the complexity of data has been increasing owing to the advancement of information technology (IT). Big data analysts and engineers have therefore been actively conducting research to minimize the analysis targets for faster processing and analysis of big data. Hadoop, which is widely used as a big data platform, provides various processing and analysis functions, including minimization of analysis targets through Hive, which is a subproject of Hadoop. However, Hive uses a vast amount of memory for data deduplication because it is implemented without considering the complexity of data. Therefore, an efficient algorithm has been proposed for data deduplication of complex structures. The performance evaluation results demonstrated that the proposed algorithm reduces the memory usage and data deduplication time by approximately 79% and 0.677%, respectively, compared to Hive. In the future, performance evaluation based on a large number of data nodes is required for a realistic verification of the proposed algorithm.

Big-Data Traffic Analysis for the Campus Network Resource Efficiency (학내 망 자원 효율화를 위한 빅 데이터 트래픽 분석)

  • An, Hyun-Min;Lee, Su-Kang;Sim, Kyu-Seok;Kim, Ik-Han;Jin, Seo-Hoon;Kim, Myung-Sup
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.40 no.3
    • /
    • pp.541-550
    • /
    • 2015
  • The importance of efficient enterprise network management has been emphasized continuously because of the rapid utilization of Internet in a limited resource environment. For the efficient network management, the management policy that reflects the characteristics of a specific network extracted from long-term traffic analysis is essential. However, the long-term traffic data could not be handled in the past and there was only simple analysis with the shot-term traffic data. However, as the big data analytics platforms are developed, the long-term traffic data can be analyzed easily. Recently, enterprise network resource efficiency through the long-term traffic analysis is required. In this paper, we propose the methods of collecting, storing and managing the long-term enterprise traffic data. We define several classification categories, and propose a novel network resource efficiency through the multidirectional statistical analysis of classified long-term traffic. The proposed method adopted to the campus network for the evaluation. The analysis results shows that, for the efficient enterprise network management, the QoS policy must be adopted in different rules that is tuned by time, space, and the purpose.

Study of MongoDB Architecture by Data Complexity for Big Data Analysis System (빅데이터 분석 시스템 구현을 위한 데이터 구조의 복잡성에 따른 MongoDB 환경 구성 연구)

  • Hyeopgeon Lee;Young-Woon Kim;Jin-Woo Lee;Seong Hyun Lee
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.16 no.5
    • /
    • pp.354-361
    • /
    • 2023
  • Big data analysis systems apply NoSQL databases like MongoDB to store, process, and analyze diverse forms of large-scale data. MongoDB offers scalability and fast data processing speeds through distributed processing and data replication, depending on its configuration. This paper investigates the suitable MongoDB environment configurations for implementing big data analysis systems. For performance evaluation, we configured both single-node and multi-node environments. In the multi-node setup, we expanded the number of data nodes from two to three and measured the performance in each environment. According to the analysis, the processing speeds for complex data structures with three or more dimensions are approximately 5.75% faster in the single-node environment compared to an environment with two data nodes. However, a setting with three data nodes processes data about 25.15% faster than the single-node environment. On the other hand, for simple one-dimensional data structures, the multi-node environment processes data approximately 28.63% faster than the single-node environment. Further research is needed to practically validate these findings with diverse data structures and large volumes of data.

Web Crawler Service Implementation for Information Retrieval based on Big Data Analysis (빅데이터 분석 기반의 정보 검색을 위한 웹 크롤러 서비스 구현)

  • Kim, Hye-Suk;Han, Na;Lim, Suk-Ja
    • Journal of Digital Contents Society
    • /
    • v.18 no.5
    • /
    • pp.933-942
    • /
    • 2017
  • In this paper, we propose a web crawler service method for collecting information efficiently about college students and job-seeker's external activities, competition, and scholarship. The proposed web crawler service uses Jsoup tree analysis and Json format data transmission method to avoid problems of duplicated crawling while crawling at high speed. After collecting relevant information for 24 hours, we were able to confirm that the web crawler service is running with an accuracy of 100%. It is expected that the web crawler service can be applied to various web sites in the future to improve the web crawler service.

A Study on Building a Test Bed for Smart Manufacturing Technology (스마트 제조기술을 위한 테스트베드 구축에 관한 연구)

  • Cho, Choon-Nam
    • Journal of the Korean Institute of Electrical and Electronic Material Engineers
    • /
    • v.34 no.6
    • /
    • pp.475-479
    • /
    • 2021
  • There are many difficulties in the applications of smart manufacturing technology in the era of the 4th industrial revolution. In this paper, a test bed was built to aim for acquiring smart manufacturing technology, and the test bed was designed to acquire basic technologies necessary for PLC (Programmable Logic Controller), HMI, Internet of Things (IoT), artificial intelligence (AI) and big data. By building a vehicle maintenance lift that can be easily accessed by the general public, PLC control technology and HMI drawing technology can be acquired, and by using cloud services, workers can respond to emergencies and alarms regardless of time and space. In addition, by managing and monitoring data for smart manufacturing, it is possible to acquire basic technologies necessary for embedded systems, the Internet of Things, artificial intelligence, and big data. It is expected that the improvement of smart manufacturing technology capability according to the results of this study will contribute to the effect of creating added value according to the applications of smart manufacturing technology in the future.

Design of Real-Time Vehicle Information Management Platform Using an IoT-based Gateway (IoT기반 게이트웨이를 활용한 실시간 차량 정보 관리 플랫폼 설계)

  • Chang, Moon-Soo;Lee, Jeong-Il
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2018.10a
    • /
    • pp.548-551
    • /
    • 2018
  • Most vehicles are in the form of maintenance when a problem occurs by the user himself or herself. During maintenance, users are not able to operate the car while it is being serviced, and if the target vehicle is a revenue-generating vehicle, they will have to bear economic losses. Collecting vehicle information in real time, identifying problems that could arise with a vehicle based on the collected big data and providing advance service rather than after-sales service can help secure vehicle operation and reduce economic loss. Thus, in this thesis, a platform was designed to design IoT-based gateways, collect real-time vehicle information, and organize big data to provide vehicle information in real time.

  • PDF