• Title/Summary/Keyword: Big-data Software

Search Result 441, Processing Time 0.028 seconds

Implementation of High Speed Big Data Processing System using In Memory Data Grid in Semiconductor Process (반도체 공정에서 인 메모리 데이터 그리드를 이용한 고속의 빅데이터 처리 시스템 구현)

  • Park, Jong-Beom;Lee, Alex;Kim, Tony
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.15 no.5
    • /
    • pp.125-133
    • /
    • 2016
  • Data processing capacity and speed are rapidly increasing due to the development of hardware and software in recent time. As a result, data usage is geometrically increasing and the amount of data which computers have to process has already exceeded five-thousand transaction per second. That is, the importance of Big Data is due to its 'real-time' and this makes it possible to analyze all the data in order to obtain accurate data at right time under any circumstances. Moreover, there are many researches about this as construction of smart factory with the application of Big Data is expected to have reduction in development, production, and quality management cost. In this paper, system using In-Memory Data Grid for high speed processing is implemented in semiconductor process which numerous data occur and improved performance is proven with experiments. Implemented system is expected to be possible to apply on not only the semiconductor but also any fields using Big Data and further researches will be made for possible application on other fields.

Performing Data Integration: Handed-code Approach vs. Tool-based Approach

  • Koo, Heung-Seo
    • Journal of the Korea Society of Computer and Information
    • /
    • v.24 no.7
    • /
    • pp.39-44
    • /
    • 2019
  • Data integration technology is one of the key elements in building data warehouses or big data, and is used to combine data from multiple sources and provide an integrated view to users. Traditionally, the performance of data integration uses a handed-code approach or a tool-based approach that utilizes data integration tools such as ETL. There is a debate about which methods are efficient. This study is conducted to give practitioners preparing for a data integration project an insight into how to perform data integration. This paper examines the views of experts on the controversy over the adoption of ETL tools that have been on the agenda of the data integration area for over a decade.

Panic Disorder Intelligent Health System based on IoT and Context-aware

  • Huan, Meng;Kang, Yun-Jeong;Lee, Sang-won;Choi, Dong-Oun
    • International journal of advanced smart convergence
    • /
    • v.10 no.2
    • /
    • pp.21-30
    • /
    • 2021
  • With the rapid development of artificial intelligence and big data, a lot of medical data is effectively used, and the diagnosis and analysis of diseases has entered the era of intelligence. With the increasing public health awareness, ordinary citizens have also put forward new demands for panic disorder health services. Specifically, people hope to predict the risk of panic disorder as soon as possible and grasp their own condition without leaving home. Against this backdrop, the smart health industry comes into being. In the Internet age, a lot of panic disorder health data has been accumulated, such as diagnostic records, medical record information and electronic files. At the same time, various health monitoring devices emerge one after another, enabling the collection and storage of personal daily health information at any time. How to use the above data to provide people with convenient panic disorder self-assessment services and reduce the incidence of panic disorder in China has become an urgent problem to be solved. In order to solve this problem, this research applies the context awareness to the automatic diagnosis of human diseases. While helping patients find diseases early and get treatment timely, it can effectively assist doctors in making correct diagnosis of diseases and reduce the probability of misdiagnosis and missed diagnosis.

Comparison of similarity measures and community detection algorithms using collaboration filtering (협업 필터링을 사용한 유사도 기법 및 커뮤니티 검출 알고리즘 비교)

  • Ugli, Sadriddinov Ilkhomjon Rovshan;Hong, Minpyo;Park, Doo-Soon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2022.05a
    • /
    • pp.366-369
    • /
    • 2022
  • The glut of information aggravated the process of data analysis and other procedures including data mining. Many algorithms were devised in Big Data and Data Mining to solve such an intricate problem. In this paper, we conducted research about the comparison of several similarity measures and community detection algorithms in collaborative filtering for movie recommendation systems. Movielense data set was used to do an empirical experiment. We applied three different similarity measures: Cosine, Euclidean, and Pearson. Moreover, betweenness and eigenvector centrality were used to detect communities from the network. As a result, we elucidated which algorithm is more suitable than its counterpart in terms of recommendation accuracy.

Big Data Analysis of Software Performance Trend using SPC with Flexible Moving Window and Fuzzy Theory (가변 윈도우 기법을 적용한 통계적 공정 제어와 퍼지추론 기법을 이용한 소프트웨어 성능 변화의 빅 데이터 분석)

  • Lee, Dong-Hun;Park, Jong-Jin
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.18 no.11
    • /
    • pp.997-1004
    • /
    • 2012
  • In enterprise software projects, performance issues have become more critical during recent decades. While developing software products, many performance tests are executed in the earlier development phase against the newly added code pieces to detect possible performance regressions. In our previous research, we introduced the framework to enable automated performance anomaly detection and reduce the analysis overhead for identifying the root causes, and showed Statistical Process Control (SPC) can be successfully applied to anomaly detection. In this paper, we explain the special performance trend in which the existing anomaly detection system can hardly detect the noticeable performance change especially when a performance regression is introduced and recovered again a while later. Within the fixed number of sampling period, the fluctuation gets aggravated and the lower and upper control limit get relaxed so that sometimes the existing system hardly detect the noticeable performance change. To resolve the issue, we apply dynamically tuned sampling window size based on the performance trend, and Fuzzy theory to find an appropriate size of the moving window.

Design of Efficient Big Data Collection Method based on Mass IoT devices (방대한 IoT 장치 기반 환경에서 효율적인 빅데이터 수집 기법 설계)

  • Choi, Jongseok;Shin, Yongtae
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.14 no.4
    • /
    • pp.300-306
    • /
    • 2021
  • Due to the development of IT technology, hardware technologies applied to IoT equipment have recently been developed, so smart systems using low-cost, high-performance RF and computing devices are being developed. However, in the infrastructure environment where a large amount of IoT devices are installed, big data collection causes a load on the collection server due to a bottleneck between the transmitted data. As a result, data transmitted to the data collection server causes packet loss and reduced data throughput. Therefore, there is a need for an efficient big data collection technique in an infrastructure environment where a large amount of IoT devices are installed. Therefore, in this paper, we propose an efficient big data collection technique in an infrastructure environment where a vast amount of IoT devices are installed. As a result of the performance evaluation, the packet loss and data throughput of the proposed technique are completed without loss of the transmitted file. In the future, the system needs to be implemented based on this design.

Frequency and Social Network Analysis of the Bible Data using Big Data Analytics Tools R (빅데이터 분석도구 R을 이용한 성경 데이터의 빈도와 소셜 네트워크 분석)

  • Ban, ChaeHoon;Ha, JongSoo;Kim, Dong Hyun
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.24 no.2
    • /
    • pp.166-171
    • /
    • 2020
  • Big data processing technology that can store and analyze data and obtain new knowledge has been adjusted for importance in many fields of the society. Big data is emerging as an important problem in the field of information and communication technology, but the mind of continuous technology is rising. the R, a tool that can analyze big data, is a language and environment that enables information analysis of statistical bases. In this paper, we use this to analyze the Bible data. We analyze the four Gospels of the New Testament in the Bible. We collect the Bible data and perform filtering for analysis. The R is used to investigate the frequency of what text is distributed and analyze the Bible through social network analysis, in which words from a sentence are paired and analyzed between words for accurate data analysis.

A Quality Evaluation Model for Distributed Processing Systems of Big Data (빅데이터 분산처리시스템의 품질평가모델)

  • Choi, Seung-Jun;Park, Jea-Won;Kim, Jong-Bae;Choi, Jae-Hyun
    • Journal of Digital Contents Society
    • /
    • v.15 no.4
    • /
    • pp.533-545
    • /
    • 2014
  • According to the evolving of IT technologies, the amount of data we are facing increasing exponentially. Thus, the technique for managing and analyzing these vast data that has emerged is a distributed processing system of big data. A quality evaluation for the existing distributed processing systems has been proceeded by the structured data environment. Thus, if we apply this to the evaluation of distributed processing systems of big data which has to focus on the analysis of the unstructured data, a precise quality assessment cannot be made. Therefore, a study of the quality evaluation model for the distributed processing systems is needed, which considers the environment of the analysis of big data. In this paper, we propose a new quality evaluation model by deriving the quality evaluation elements based on the ISO/IEC9126 which is the international standard on software quality, and defining metrics for validating the elements.

Blockchain and AI-based big data processing techniques for sustainable agricultural environments (지속가능한 농업 환경을 위한 블록체인과 AI 기반 빅 데이터 처리 기법)

  • Yoon-Su Jeong
    • Advanced Industrial SCIence
    • /
    • v.3 no.2
    • /
    • pp.17-22
    • /
    • 2024
  • Recently, as the ICT field has been used in various environments, it has become possible to analyze pests by crops, use robots when harvesting crops, and predict by big data by utilizing ICT technologies in a sustainable agricultural environment. However, in a sustainable agricultural environment, efforts to solve resource depletion, agricultural population decline, poverty increase, and environmental destruction are constantly being demanded. This paper proposes an artificial intelligence-based big data processing analysis method to reduce the production cost and increase the efficiency of crops based on a sustainable agricultural environment. The proposed technique strengthens the security and reliability of data by processing big data of crops combined with AI, and enables better decision-making and business value extraction. It can lead to innovative changes in various industries and fields and promote the development of data-oriented business models. During the experiment, the proposed technique gave an accurate answer to only a small amount of data, and at a farm site where it is difficult to tag the correct answer one by one, the performance similar to that of learning with a large amount of correct answer data (with an error rate within 0.05) was found.

A Decision Tree-based Music Recommendation System Using the user experience (사용자 경험정보를 고려한 결정트리 기반 음악 추천 시스템)

  • Kim, Yu-ri;Kim, Seong-gi;Kim, Jeong-Ho;Jo, Jae-rim;Lee, Dong-wook;Kim, Seok-Jin;Jeon, Soo-bin;Seo, Dong-mahn
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2020.11a
    • /
    • pp.655-658
    • /
    • 2020
  • 최근 IT 기술의 발달로 태블릿, 스마트폰과 같은 다양한 디바이스로 손쉽게 음악을 감상할 수 있다. 하지만 최근 이런 기술 발달과는 다르게 사용자가 원하는 음악을 검색하는 방법은 고전적인 형태에서 벗어나지 않고 있다. 기존 음악 검색 방법은 텍스트 기반, 내용 기반, 소비자 감성 기반의 음악 추천 검색 방법이 있으며 저장된 메타 데이터를 이용하여 사용자의 질의에 대한 결과만 제공할 뿐 사용자의 경험 정보를 고려하지 않는다. 그리고 기존 플랫폼들은 사용자가 최근 많이 들은 가수, 장르, 분위기를 종합하여 사용자에게 어울리는 음악을 추천을 할 뿐 사용자의 경험정보를 고려하여 음악을 추천하지는 않는다. 본 논문에서는 사용자의 경험 정보를 활용하여 사용자 맞춤형 음악 추천 시스템을 제안한다. 본 시스템은 사용자의 현재 기분 정보, 주변 날씨 정보 등을 입력 받는다. 이후, 경험 정보를 기반으로 결정 트리를 통해 사용자 요구 기반의 음악 추천 시스템을 구축하였다.