• 제목/요약/키워드: Big Data Structure

검색결과 383건 처리시간 0.029초

Wavelet-like convolutional neural network structure for time-series data classification

  • Park, Seungtae;Jeong, Haedong;Min, Hyungcheol;Lee, Hojin;Lee, Seungchul
    • Smart Structures and Systems
    • /
    • 제22권2호
    • /
    • pp.175-183
    • /
    • 2018
  • Time-series data often contain one of the most valuable pieces of information in many fields including manufacturing. Because time-series data are relatively cheap to acquire, they (e.g., vibration signals) have become a crucial part of big data even in manufacturing shop floors. Recently, deep-learning models have shown state-of-art performance for analyzing big data because of their sophisticated structures and considerable computational power. Traditional models for a machinery-monitoring system have highly relied on features selected by human experts. In addition, the representational power of such models fails as the data distribution becomes complicated. On the other hand, deep-learning models automatically select highly abstracted features during the optimization process, and their representational power is better than that of traditional neural network models. However, the applicability of deep-learning models to the field of prognostics and health management (PHM) has not been well investigated yet. This study integrates the "residual fitting" mechanism inherently embedded in the wavelet transform into the convolutional neural network deep-learning structure. As a result, the architecture combines a signal smoother and classification procedures into a single model. Validation results from rotor vibration data demonstrate that our model outperforms all other off-the-shelf feature-based models.

Text Classification on Social Network Platforms Based on Deep Learning Models

  • YA, Chen;Tan, Juan;Hoekyung, Jung
    • Journal of information and communication convergence engineering
    • /
    • 제21권1호
    • /
    • pp.9-16
    • /
    • 2023
  • The natural language on social network platforms has a certain front-to-back dependency in structure, and the direct conversion of Chinese text into a vector makes the dimensionality very high, thereby resulting in the low accuracy of existing text classification methods. To this end, this study establishes a deep learning model that combines a big data ultra-deep convolutional neural network (UDCNN) and long short-term memory network (LSTM). The deep structure of UDCNN is used to extract the features of text vector classification. The LSTM stores historical information to extract the context dependency of long texts, and word embedding is introduced to convert the text into low-dimensional vectors. Experiments are conducted on the social network platforms Sogou corpus and the University HowNet Chinese corpus. The research results show that compared with CNN + rand, LSTM, and other models, the neural network deep learning hybrid model can effectively improve the accuracy of text classification.

Exploring the dynamic knowledge structure of studies on the Internet of things: Keyword analysis

  • Yoon, Young Seog;Zo, Hangjung;Choi, Munkee;Lee, Donghyun;Lee, Hyun-woo
    • ETRI Journal
    • /
    • 제40권6호
    • /
    • pp.745-758
    • /
    • 2018
  • A wide range of studies in various disciplines has focused on the Internet of Things (IoT) and cyber-physical systems (CPS). However, it is necessary to summarize the current status and to establish future directions because each study has its own individual goals independent of the completion of all IoT applications. The absence of a comprehensive understanding of IoT and CPS has disrupted an efficient resource allocation. To assess changes in the knowledge structure and emerging technologies, this study explores the dynamic research trends in IoT by analyzing bibliographic data. We retrieved 54,237 keywords in 12,600 IoT studies from the Scopus database, and conducted keyword frequency, co-occurrence, and growth-rate analyses. The analysis results reveal how IoT technologies have been developed and how they are connected to each other. We also show that such technologies have diverged and converged simultaneously, and that the emerging keywords of trust, smart home, cloud, authentication, context-aware, and big data have been extracted. We also unveil that the CPS is directly involved in network, security, management, cloud, big data, system, industry, architecture, and the Internet.

다중 레이어 구조로 된 보안 파일 포맷 및 API 설계 (File Formats with a Multi-Layer Structure and API Design)

  • 박종문;윤정호;조현태;김기창
    • 한국정보통신학회:학술대회논문집
    • /
    • 한국정보통신학회 2012년도 추계학술대회
    • /
    • pp.123-127
    • /
    • 2012
  • 컴퓨터와 인터넷의 보급과 더불어 스마트폰의 확산으로 인하여, 하루에도 수많은 데이터가 생산되고 수정되고 있다. 이렇게 데이터양의 증가로 인하여 이를 안전하게 저장하는 방법이 새로운 문제로 떠오르고 있다. 이에 본 논문에서는, 계층 데이터 구조로 이루어진 다중 레이어 방식으로 빅 데이터를 저장하며 레이어별 암호화를 적용하는 새로운 보안 파일 Format과 이를 이용할 수 있는 API를 소개하고자 한다. 본 논문에서 소개하는 새로운 보안 파일 Format은 앞으로 많은 분야에 적용되어 사용되기를 기대한다.

  • PDF

조달청 OPEN API 빅데이터를 활용한 공공 소프트웨어 산업의 SNA 패턴 분석 (SNA Pattern Analysis on the Public Software Industry based on Open API Big Data from Korea Public Procurement Service)

  • 김소정;심선영;서용원
    • 정보화정책
    • /
    • 제24권3호
    • /
    • pp.42-66
    • /
    • 2017
  • 본 연구는 우리나라를 대표하는 개방 데이터인 조달청의 빅데이터를 활용하여, 최근 사회과학 연구에서 활발하게 사용되는 사회관계망 분석을 통해 정부의 특정 정책(소프트웨어 대기업 참여 상한제) 전후의 산업 네트워크 구조를 비교 분석함으로써 소프트웨어 시장의 생태계 변화를 조망하고 공공데이터 개방의 시사점을 살펴보는 것을 목적으로 한다. 2013년에서 2015년까지 3년에 걸쳐 공공 소프트웨어 시장의 정보화 사업에 대한 발주 및 수주 계약 데이터를 분석해 본 결과, 첫째 공공 소프트웨어 시장에서 Power Law현상이 관찰되고 있으며, 이 현상은 규제 등의 외부적 충격과 상관없이 지속되고 있음을 알 수 있었다. 둘째, 이 시장에서 Power Law현상은 지속되고 있었지만 생태계의 구성은 년도별로 유의미한 차이를 보임도 확인하였다. 이러한 결과를 바탕으로, 공공 소프트웨어 시장의 생태계 구성 및 변화에 대한 시사점을 도출하고, 근본적으로 이러한 분석을 가능케 하는 공공 빅 데이터 개방의 장점에 대해 논의하였다.

용접 빅데이터 환경에서 상관분석 및 회귀분석을 이용한 작업 패턴 분석 모형에 관한 연구 (A Study on a Working Pattern Analysis Prototype using Correlation Analysis and Linear Regression Analysis in Welding BigData Environment)

  • 정세훈;심춘보
    • 한국전자통신학회논문지
    • /
    • 제9권10호
    • /
    • pp.1071-1078
    • /
    • 2014
  • 최근 빅데이터(Big Data)를 이용한 정보 제공 서비스가 확대되고 빅데이터 처리 기술 역시 IT 업체의 중요한 이슈로 학문적인 연구가 활발히 진행되고 있는 실정이다. 이에 본 논문에서는 R 프로그래밍을 기반으로 용접의 빅데이터 분석 및 추출을 통하여 용접사의 숙련된 패턴을 분석하고 분석된 결과를 비 숙련공에게 제공함으로써 용접 품질 및 용접 시간 단축 등의 용접 작업에 적용되는 비용을 절감하고자 한다. 용접은 숙련공이 되기 위하여 오랜 시간을 투자해야 하는 문제점이 있다. 이러한 단점을 해결하고자 숙련공들의 용접 패턴 분석을 위하여 다량의 패턴 변수에 R의 연관 규칙 알고리즘과 회귀분석 방식을 적용한다. 상위 N개의 규칙을 분석한 후 분석된 규칙의 변수에 따른 숙련자의 패턴을 분석한다. 본 논문에서는 분석된 용접 패턴 분석을 통해 실험 결과를 분석하여 전력소비량과 와이어 소모 길이에 대한 패턴 구조를 확인하였다.

빅데이터 분석 적용을 통한 공정 최적화 사례연구: LCD 공정 품질분석을 중심으로 (A Case Study on Product Production Process Optimization using Big Data Analysis: Focusing on the Quality Management of LCD Production)

  • 박종태;이상곤
    • 한국IT서비스학회지
    • /
    • 제21권2호
    • /
    • pp.97-107
    • /
    • 2022
  • Recently, interest in smart factories is increasing. Investments to improve intelligence/automation are also being made continuously in manufacturing plants. Facility automation based on sensor data collection is now essential. In addition, we are operating our factories based on data generated in all areas of production, including production management, facility operation, and quality management, and an integrated standard information system. When producing LCD polarizer products, it is most important to link trace information between data generated by individual production processes. All systems involved in production must ensure that there is no data loss and data integrity is ensured. The large-capacity data collected from individual systems is composed of key values linked to each other. A real-time quality analysis processing system based on connected integrated system data is required. In this study, large-capacity data collection, storage, integration and loss prevention methods were presented for optimization of LCD polarizer production. The identification Risk model of inspection products can be added, and the applicable product model is designed to be continuously expanded. A quality inspection and analysis system that maximizes the yield rate was designed by using the final inspection image of the product using big data technology. In the case of products that are predefined as analysable products, it is designed to be verified with the big data knn analysis model, and individual analysis results are continuously applied to the actual production site to operate in a virtuous cycle structure. Production Optimization was performed by applying it to the currently produced LCD polarizer production line.

복잡한 구조의 데이터 중복제거를 위한 효율적인 알고리즘 연구 (Study of Efficient Algorithm for Deduplication of Complex Structure)

  • 이협건;김영운;김기영
    • 한국정보전자통신기술학회논문지
    • /
    • 제14권1호
    • /
    • pp.29-36
    • /
    • 2021
  • IT기술의 발달로 인해 발생되는 데이터양은 기하급수적으로 급격하게 증가하고 있으며, 데이터 구조의 복잡성은 높아지고 있다. 빅데이터 분석가와 빅데이터 엔지니어들은 이러한 빅데이터들을 보다 빠르게 데이터 처리 및 데이터 분석을 수행을 목표로 분석 대상의 데이터양을 최소화하기 위한 연구가 기업 및 가관 등 활발하게 이뤄지고 있다. 빅데이터 플랫폼으로 많이 활용되는 하둡은 서브프로젝트인 Hive를 통해 분석 대상의 데이터 최소화 등 다양한 데이터 처리 및 데이터 분석 기능을 제공하고 있다. 그러나 Hive는 데이터의 복잡성을 고려하지 않고 구현되어 중복 제거에 방대한 양의 메모리를 사용한다. 이에 복잡한 구조의 데이터 중복제거를 위한 효율적인 알고리즘을 제안한다. 성능평가 결과, 제안하는 알고리즘은 Hive에 비해 메모리 사용량은 최대 79%, 데이터 중복제거 시간은 0.677% 감소한다. 향후, 제안하는 알고리즘의 현실적인 검증을 위해 다수의 데이터 노드 기반 성능 평가가 필요하다.

도시 열환경 분석을 위한 공간정보 빅데이터 구축 (Construction of Spatial Information Big Data for Urban Thermal Environment Analysis)

  • 이준호;윤성환
    • 대한건축학회논문집:계획계
    • /
    • 제36권5호
    • /
    • pp.53-58
    • /
    • 2020
  • The purpose of this study is to build a database of Spatial information Bigdata of cities using satellite images and spatial information, and to examine the correlations with the surface temperature. Using architectural structure and usage in building information, DEM and Slope topographical information for constructed with 300 × 300 mesh grids for Busan. The satellite image is used to prepare the Normalized Difference Built-up Index (NDBI), Normalized Difference Vegetation Index (NDVI), Bare Soil Index (BI), and Land Surface Temperature (LST). In addition, the building area in the grid was calculated and the building ratio was constructed to build the urban environment DB. In architectural structure, positive correlation was found in masonry and concrete structures. On the terrain, negative correlations were observed between DEM and slope. NDBI and BI were positively correlated, and NDVI was negatively correlated. The higher the Building ratio, the higher the surface temperature. It was found that the urban environment DB could be used as a basic data for urban environment analysis, and it was possible to quantitatively grasp the impact on the architecture and urban environment by adding local meteorological factors. This result is expected to be used as basic data for future urban environment planning and disaster prevention data construction.

APT 공격 탐지를 위한 공격 경로 및 의도 인지 시스템 (Attack Path and Intention Recognition System for detecting APT Attack)

  • 김남욱;엄정호
    • 디지털산업정보학회논문지
    • /
    • 제16권1호
    • /
    • pp.67-78
    • /
    • 2020
  • Typical security solutions such as intrusion detection system are not suitable for detecting advanced persistent attack(APT), because they cannot draw the big picture from trivial events of security solutions. Researches on techniques for detecting multiple stage attacks by analyzing the correlations between security events or alerts are being actively conducted in academic field. However, these studies still use events from existing security system, and there is insufficient research on the structure of the entire security system suitable for advanced persistent attacks. In this paper, we propose an attack path and intention recognition system suitable for multiple stage attacks like advanced persistent attack detection. The proposed system defines the trace format and overall structure of the system that detects APT attacks based on the correlation and behavior analysis, and is designed with a structure of detection system using deep learning and big data technology, etc.