• 제목/요약/키워드: Big Data Environment

검색결과 962건 처리시간 0.033초

IoT 환경에서 센서 데이터 처리율 향상을 위한 Apriori 기반 빅데이터 처리 시스템 (Apriori Based Big Data Processing System for Improve Sensor Data Throughput in IoT Environments)

  • 송진수;김수진;신용태
    • 정보처리학회논문지:컴퓨터 및 통신 시스템
    • /
    • 제10권10호
    • /
    • pp.277-284
    • /
    • 2021
  • 최근 스마트 홈 환경은 무선 정보통신 기술과 융합을 통해서 다양한 데이터를 수집·통합·활용하는 플랫폼이 될 것으로 전망되고 있으며 실제로 스마트 홈 내부에는 다양한 센서를 탑재한 스마트 디바이스 수가 점점 증가하고 있다. 증가된 스마트 디바이스 수만큼 처리해야하는 데이터의 양도 증가하고 있으며 이를 효과적으로 처리하기 위해 빅데이터 처리 시스템이 활발하게 도입되고 있다. 그러나 기존 빅데이터 처리 시스템은 분산 노드에 할당되기 전 모든 요청이 클러스터 드라이버로 향하기 때문에 동시에 많은 요청이 발생하는 경우 분할 작업을 관리하는 클러스터 드라이버에 병목현상이 발생하고, 이는 네트워크를 공유하는 클러스터 전체의 성능감소로 이어진다. 특히 작은 데이터 처리를 지속해서 요청하는 스마트 홈 디바이스에서 지연율이 더 크게 나타난다. 이에 본 논문에서는 동시에 다수의 센서에서 요청이 발생하는 스마트 홈 환경에서 효과적인 데이터 처리를 위한 Apriori 기반 빅데이터 시스템을 설계하였다. 제안하는 시스템의 성능평가 결과에 따르면, 데이터 처리 시간은 기존 시스템에 비해 최소 19.2%에서 최대 38.6% 단축됐다. 이러한 결과가 발생한 이유는 측정되는 데이터의 형태와 관련이 있다. 스마트 홈 환경은 수집되는 데이터의 양은 방대하나 각 데이터의 용량은 작기 때문에 캐시 서버의 사용이 데이터 처리에 큰 역할을 하며, Apriori 알고리즘을 통한 연관도 분석으로 사용자의 행동 습관과 연관도가 높은 센서 데이터를 캐시에 저장하기 때문에 캐시 서버의 활용률이 매우 높다.

빅데이터 분석을 이용한 디지털 패션 테크에 대한 인식 연구 (Perceptions and Trends of Digital Fashion Technology - A Big Data Analysis -)

  • 송은영;임호선
    • 한국의류산업학회지
    • /
    • 제23권3호
    • /
    • pp.380-389
    • /
    • 2021
  • This study aimed to reveal the perceptions and trends of digital fashion technology through an informational approach. A big data analysis was conducted after collecting the text shown in a web environment from April 2019 to April 2021. Key words were derived through text mining analysis and network analysis, and the structure of perception of digital fashion technology was identified. Using textoms, we collected 8144 texts after data refinement, conducted a frequency of emergence and central component analysis, and visualized the results with word cloud and N-gram. The frequency of appearance also generated matrices with the top 70 words, and a structural equivalent analysis was performed. The results were presented with network visualizations and dendrograms. Fashion, digital, and technology were the most frequently mentioned topics, and the frequencies of platform, digital transformation, and start-ups were also high. Through clustering, four clusters of marketing were formed using fashion, digital technology, startups, and augmented reality/virtual reality technology. Future research on startups and smart factories with technologies based on stable platforms is needed. The results of this study contribute to increasing the fashion industry's knowledge on digital fashion technology and can be used as a foundational study for the development of research on related topics.

A Study on the Consumer Perception and Keyword Analysis of Meal-kit Using Big Data

  • Jung, Sunmi;Ryu, Gihwan;Lim, Jeongsook;Kim, Heeyoung
    • International Journal of Internet, Broadcasting and Communication
    • /
    • 제14권2호
    • /
    • pp.206-211
    • /
    • 2022
  • As the level of consumption is improved and cultural life is pursued, the consumer's consciousness structure is rapidly changing, and the demand for product selection level, variety, and quality is becoming more diverse. The restaurant economy is falling due to the prolonged COVID-19, the economic recession, income decline, and changes in population structure and lifestyle, but the Meal- kit market is growing rapidly. This study aims to identify the consumer perception of Meal-kit, which is rapidly growing as an alternative to existing meals in the fields of dining out, food, and distribution due to the development of technology and social environment using big data. As a result of the analysis, the keywords with the highest frequency of appearance were in the order of Meal-kit, Cooking, Product, Launching, and Market and were divided into 8 groups through the CONCOR analysis. We want to identify consumer trends related to the key keywords of Meal-kit, present effective data related to Meal-kit demand for Meal-kit specialized companies, and provide implications for establishing marketing strategies for differentiated competitive advantage.

Applying a big data analysis to evaluate the suitability of shelter locations for the evacuation of residents in case of radiological emergencies

  • Jin Sik Choi;Jae Wook Kim;Han Young Joo;Joo Hyun Moon
    • Nuclear Engineering and Technology
    • /
    • 제55권1호
    • /
    • pp.261-269
    • /
    • 2023
  • During a nuclear power plant (NPP) accident, radioactive material may be released into the surrounding environment in the form of a radioactive plume. The behavior of the radioactive plume is influenced by meteorological factors such as wind direction and speed. If the residents are evacuated to a shelter in the direction of the flow of the radioactive plume, the radiation exposure of the residents may increase, contrary to the purpose of the evacuation. To avoid such an undesirable outcome, this paper applies a big data analysis to evaluate the suitability of the shelter locations near 5 NPPs in the Republic of Korea in terms of the seasonal wind direction frequency in those areas. To this end, the wind data measured around the NPPs from 2016 to 2020 were analyzed to derive the seasonal wind direction frequency using a big data analysis. These analyses results were then used to determine how many shelters around NPPs locate in areas with prevailing wind direction per season. Then, suggestions were made on the direction for residents not to evacuate, if possible, that is, the prevailing seasonal wind directions for 5 NPPs, depending on the season in which the accident occurs.

A Deep Learning Approach for Intrusion Detection

  • Roua Dhahbi;Farah Jemili
    • International Journal of Computer Science & Network Security
    • /
    • 제23권10호
    • /
    • pp.89-96
    • /
    • 2023
  • Intrusion detection has been widely studied in both industry and academia, but cybersecurity analysts always want more accuracy and global threat analysis to secure their systems in cyberspace. Big data represent the great challenge of intrusion detection systems, making it hard to monitor and analyze this large volume of data using traditional techniques. Recently, deep learning has been emerged as a new approach which enables the use of Big Data with a low training time and high accuracy rate. In this paper, we propose an approach of an IDS based on cloud computing and the integration of big data and deep learning techniques to detect different attacks as early as possible. To demonstrate the efficacy of this system, we implement the proposed system within Microsoft Azure Cloud, as it provides both processing power and storage capabilities, using a convolutional neural network (CNN-IDS) with the distributed computing environment Apache Spark, integrated with Keras Deep Learning Library. We study the performance of the model in two categories of classification (binary and multiclass) using CSE-CIC-IDS2018 dataset. Our system showed a great performance due to the integration of deep learning technique and Apache Spark engine.

클라우드 기반의 공개의료 빅데이터 분석을 통한 삶의 질에 영향을 미치는 요인분석 (An Analysis of Factors Affecting Quality of Life through the Analysis of Public Health Big Data)

  • 김민경;조영복
    • 한국정보통신학회논문지
    • /
    • 제22권6호
    • /
    • pp.835-841
    • /
    • 2018
  • 본 연구에서 공개 의료 빅데이터 분석을 지역사회건강조사 2012~2014년 자료를 이용해 개인의 건강관련 삶의 질 차이와 삶의 질에 영향을 미치는 요인을 분석하였다. 제안논문에서는 공개의료 빅데이터 분석을 위해 Hadoop 기반의 Spack을 이용해 병렬처리 지원을 위한 클라우드 메니저를 구성하고 개인의 삶의 질에 영향을 미치는 요인을 하드웨어의 제약없이 빠르게 분석하였다. 건강관련 삶의 질에 미치는 영향을 개인적 특성과 지역사회 특성으로 구분하여 단계별 다수준 회귀분석(ANOVA, t-test)을 실시하였다. 연구결과 개인별 삶의 질에 영향을 미치는 요인으로는 남자 평균 73.8점, 여자 평균 70.0점으로 남자가 여자보다 건강관련 삶의 질이 높은 것으로 나타났다.

RHIPE 플랫폼에서 빅데이터 로지스틱 회귀를 위한 학습 알고리즘 (Learning algorithms for big data logistic regression on RHIPE platform)

  • 정병호;임동훈
    • Journal of the Korean Data and Information Science Society
    • /
    • 제27권4호
    • /
    • pp.911-923
    • /
    • 2016
  • 빅데이터 시대에 머신러닝의 중요성은 더욱 부각되고 있고 로지스틱 회귀는 머신러닝에서 분류를 위한 방법으로 의료, 경제학, 마케팅 및 사회과학 전반에 걸쳐 널리 사용되고 있다. 지금까지 R과 Hadoop의 통합환경인 RHIPE 플랫폼은 설치 및 MapReduce 구현의 어려움으로 인해 거의 연구가 이루지 지지 않았다. 본 논문에서는 대용량 데이터에 대해 로지스틱 회귀 추정을 위한 두가지 알고리즘 즉, Gradient Descent 알고리즘과 Newton-Raphson 알고리즘에 대해 MapReduce로 구현하고, 실제 데이터와 모의실험 데이터를 가지고 이들 알고리즘 간의 성능을 비교하고자 한다. 알고리즘 성능 실험에서 Gradient Descent 알고리즘은 학습률에 크게 의존하고 또한 데이터에 따라 수렴하지 않는 문제를 갖고 있다. Newton-Raphson 알고리즘은 학습률이 불필요 할 뿐만 아니라 모든 실험 데이터에 대해 좋은 성능을 보였다.

빅데이터 분석을 활용한 스마트팩토리 연구 동향 분석 (Analysis of Smart Factory Research Trends Based on Big Data Analysis)

  • 이은지;조철호
    • 품질경영학회지
    • /
    • 제49권4호
    • /
    • pp.551-567
    • /
    • 2021
  • Purpose: The purpose of this paper is to present implications by analyzing research trends on smart factories by text analysis and visual analysis(Comprehensive/ Fields / Years-based) which are big data analyses, by collecting data based on previous studies on smart factories. Methods: For the collection of analysis data, deep learning was used in the integrated search on the Academic Research Information Service (www.riss.kr) to search for "SMART FACTORY" and "Smart Factory" as search terms, and the titles and Korean abstracts were scrapped out of the extracted paper and they are organize into EXCEL. For the final step, 739 papers derived were analyzed using the Rx64 4.0.2 program and Rstudio using text mining, one of the big data analysis techniques, and Word Cloud for visualization. Results: The results of this study are as follows; Smart factory research slowed down from 2005 to 2014, but until 2019, research increased rapidly. According to the analysis by fields, smart factories were studied in the order of engineering, social science, and complex science. There were many 'engineering' fields in the early stages of smart factories, and research was expanded to 'social science'. In particular, since 2015, it has been studied in various disciplines such as 'complex studies'. Overall, in keyword analysis, the keywords such as 'technology', 'data', and 'analysis' are most likely to appear, and it was analyzed that there were some differences by fields and years. Conclusion: Government support and expert support for smart factories should be activated, and researches on technology-based strategies are needed. In the future, it is necessary to take various approaches to smart factories. If researches are conducted in consideration of the environment or energy, it is judged that bigger implications can be presented.

토픽 모형과 ChatGPT를 활용한 스마트팩토리 연관 특허 빅데이터 분석에 관한 연구 (A Study on Big Data Analysis of Related Patents in Smart Factories Using Topic Models and ChatGPT)

  • 김상국;윤민영;권태훈;임정선
    • 산업경영시스템학회지
    • /
    • 제46권4호
    • /
    • pp.15-31
    • /
    • 2023
  • In this study, we propose a novel approach to analyze big data related to patents in the field of smart factories, utilizing the Latent Dirichlet Allocation (LDA) topic modeling method and the generative artificial intelligence technology, ChatGPT. Our method includes extracting valuable insights from a large data-set of associated patents using LDA to identify latent topics and their corresponding patent documents. Additionally, we validate the suitability of the topics generated using generative AI technology and review the results with domain experts. We also employ the powerful big data analysis tool, KNIME, to preprocess and visualize the patent data, facilitating a better understanding of the global patent landscape and enabling a comparative analysis with the domestic patent environment. In order to explore quantitative and qualitative comparative advantages at this juncture, we have selected six indicators for conducting a quantitative analysis. Consequently, our approach allows us to explore the distinctive characteristics and investment directions of individual countries in the context of research and development and commercialization, based on a global-scale patent analysis in the field of smart factories. We anticipate that our findings, based on the analysis of global patent data in the field of smart factories, will serve as vital guidance for determining individual countries' directions in research and development investment. Furthermore, we propose a novel utilization of GhatGPT as a tool for validating the suitability of selected topics for policy makers who must choose topics across various scientific and technological domains.

빅데이터를 활용한 소규모 건축물 안전관리 모델에 관한 연구 (A Study on Building a Model for Safety Management of Small Buildings using Big Data)

  • 신동윤
    • 한국BIM학회 논문집
    • /
    • 제13권1호
    • /
    • pp.13-21
    • /
    • 2023
  • The purpose of this study is to establish a system that manages the safety of buildings efficiently by finding the correlation of elements related to the safety of buildings and intuitively visualizing them. Data were collected using the data of small-scale buildings managed by public institutions and the government, and an effective analysis visualization environment was established through pre-processing. We selected safety-vulnerable factors such as the structure of the building and completion date to find the relationship, and established a model to prioritize management to find vulnerable buildings.