• Title/Summary/Keyword: Big Data Structure

Search Result 383, Processing Time 0.028 seconds

Wavelet-like convolutional neural network structure for time-series data classification

  • Park, Seungtae;Jeong, Haedong;Min, Hyungcheol;Lee, Hojin;Lee, Seungchul
    • Smart Structures and Systems
    • /
    • v.22 no.2
    • /
    • pp.175-183
    • /
    • 2018
  • Time-series data often contain one of the most valuable pieces of information in many fields including manufacturing. Because time-series data are relatively cheap to acquire, they (e.g., vibration signals) have become a crucial part of big data even in manufacturing shop floors. Recently, deep-learning models have shown state-of-art performance for analyzing big data because of their sophisticated structures and considerable computational power. Traditional models for a machinery-monitoring system have highly relied on features selected by human experts. In addition, the representational power of such models fails as the data distribution becomes complicated. On the other hand, deep-learning models automatically select highly abstracted features during the optimization process, and their representational power is better than that of traditional neural network models. However, the applicability of deep-learning models to the field of prognostics and health management (PHM) has not been well investigated yet. This study integrates the "residual fitting" mechanism inherently embedded in the wavelet transform into the convolutional neural network deep-learning structure. As a result, the architecture combines a signal smoother and classification procedures into a single model. Validation results from rotor vibration data demonstrate that our model outperforms all other off-the-shelf feature-based models.

Text Classification on Social Network Platforms Based on Deep Learning Models

  • YA, Chen;Tan, Juan;Hoekyung, Jung
    • Journal of information and communication convergence engineering
    • /
    • v.21 no.1
    • /
    • pp.9-16
    • /
    • 2023
  • The natural language on social network platforms has a certain front-to-back dependency in structure, and the direct conversion of Chinese text into a vector makes the dimensionality very high, thereby resulting in the low accuracy of existing text classification methods. To this end, this study establishes a deep learning model that combines a big data ultra-deep convolutional neural network (UDCNN) and long short-term memory network (LSTM). The deep structure of UDCNN is used to extract the features of text vector classification. The LSTM stores historical information to extract the context dependency of long texts, and word embedding is introduced to convert the text into low-dimensional vectors. Experiments are conducted on the social network platforms Sogou corpus and the University HowNet Chinese corpus. The research results show that compared with CNN + rand, LSTM, and other models, the neural network deep learning hybrid model can effectively improve the accuracy of text classification.

Exploring the dynamic knowledge structure of studies on the Internet of things: Keyword analysis

  • Yoon, Young Seog;Zo, Hangjung;Choi, Munkee;Lee, Donghyun;Lee, Hyun-woo
    • ETRI Journal
    • /
    • v.40 no.6
    • /
    • pp.745-758
    • /
    • 2018
  • A wide range of studies in various disciplines has focused on the Internet of Things (IoT) and cyber-physical systems (CPS). However, it is necessary to summarize the current status and to establish future directions because each study has its own individual goals independent of the completion of all IoT applications. The absence of a comprehensive understanding of IoT and CPS has disrupted an efficient resource allocation. To assess changes in the knowledge structure and emerging technologies, this study explores the dynamic research trends in IoT by analyzing bibliographic data. We retrieved 54,237 keywords in 12,600 IoT studies from the Scopus database, and conducted keyword frequency, co-occurrence, and growth-rate analyses. The analysis results reveal how IoT technologies have been developed and how they are connected to each other. We also show that such technologies have diverged and converged simultaneously, and that the emerging keywords of trust, smart home, cloud, authentication, context-aware, and big data have been extracted. We also unveil that the CPS is directly involved in network, security, management, cloud, big data, system, industry, architecture, and the Internet.

File Formats with a Multi-Layer Structure and API Design (다중 레이어 구조로 된 보안 파일 포맷 및 API 설계)

  • Park, Jong-Moon;Yoon, Jeong-Ho;Jo, Hyeon-Tae;Kim, Ki-Chang
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2012.10a
    • /
    • pp.123-127
    • /
    • 2012
  • Since the propagation of computers and Internet along with proliferation of smartphones rise, a large amount of data is being produced and modified daily. As the usage of data soars, a way of securely storing data emerged as a new problem. In this paper, saving big-data by using hierarchical data structure with multi-layer form, to come up with new security file format and API by applying encryption on each layers, is introduced. Moreover, we expect to see shown file format in this paper to be used in various fields.

  • PDF

SNA Pattern Analysis on the Public Software Industry based on Open API Big Data from Korea Public Procurement Service (조달청 OPEN API 빅데이터를 활용한 공공 소프트웨어 산업의 SNA 패턴 분석)

  • KIM, Sojung lucia;Shim, Seon-Young;Seo, Yong-Won
    • Informatization Policy
    • /
    • v.24 no.3
    • /
    • pp.42-66
    • /
    • 2017
  • This study investigated the ecological change of public software industry, comparing the pre and post structure of industry network based on the application of the regulation restricting large company participation in public software market. For this purpose, we used big data of the software market from Korea Public Procurement Service and used the SNA(Social Network Analysis) methodology which is being actively used in the area of social science recently. Finally, we highlighted the contribution of open public data. By analyzing order and contract data of the public software industry for 3 years - from 2013 to 2015 - we found out two main things. First, we observed that Power Law distribution had been going on in the public software industry, regardless of the external impact of regulation. Second, despite the existence of such Power Law distribution, we also observed the ecological change of industry structure from year to year. We presented the implication of such findings and discussed the advantage of open public data as the original motivator of this study.

A Study on a Working Pattern Analysis Prototype using Correlation Analysis and Linear Regression Analysis in Welding BigData Environment (용접 빅데이터 환경에서 상관분석 및 회귀분석을 이용한 작업 패턴 분석 모형에 관한 연구)

  • Jung, Se-Hoon;Sim, Chun-Bo
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.9 no.10
    • /
    • pp.1071-1078
    • /
    • 2014
  • Recently, information providing service using Big Data is being expanded. Big Data processing technology is actively being academic research to an important issue in the IT industry. In this paper, we analyze a skilled pattern of welder through Big Data analysis or extraction of welding based on R programming. We are going to reduce cost on welding work including weld quality, weld operation time by providing analyzed results non-skilled welder. Welding has a problem that should be invested long time to be a skilled welder. For solving these issues, we apply connection rules algorithms and regression method to much pattern variable for welding pattern analysis of skilled welder. We analyze a pattern of skilled welder according to variable of analyzed rules by analyzing top N rules. In this paper, we confirmed the pattern structure of power consumption rate and wire consumption length through experimental results of analyzed welding pattern analysis.

A Case Study on Product Production Process Optimization using Big Data Analysis: Focusing on the Quality Management of LCD Production (빅데이터 분석 적용을 통한 공정 최적화 사례연구: LCD 공정 품질분석을 중심으로)

  • Park, Jong Tae;Lee, Sang Kon
    • Journal of Information Technology Services
    • /
    • v.21 no.2
    • /
    • pp.97-107
    • /
    • 2022
  • Recently, interest in smart factories is increasing. Investments to improve intelligence/automation are also being made continuously in manufacturing plants. Facility automation based on sensor data collection is now essential. In addition, we are operating our factories based on data generated in all areas of production, including production management, facility operation, and quality management, and an integrated standard information system. When producing LCD polarizer products, it is most important to link trace information between data generated by individual production processes. All systems involved in production must ensure that there is no data loss and data integrity is ensured. The large-capacity data collected from individual systems is composed of key values linked to each other. A real-time quality analysis processing system based on connected integrated system data is required. In this study, large-capacity data collection, storage, integration and loss prevention methods were presented for optimization of LCD polarizer production. The identification Risk model of inspection products can be added, and the applicable product model is designed to be continuously expanded. A quality inspection and analysis system that maximizes the yield rate was designed by using the final inspection image of the product using big data technology. In the case of products that are predefined as analysable products, it is designed to be verified with the big data knn analysis model, and individual analysis results are continuously applied to the actual production site to operate in a virtuous cycle structure. Production Optimization was performed by applying it to the currently produced LCD polarizer production line.

Study of Efficient Algorithm for Deduplication of Complex Structure (복잡한 구조의 데이터 중복제거를 위한 효율적인 알고리즘 연구)

  • Lee, Hyeopgeon;Kim, Young-Woon;Kim, Ki-Young
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.14 no.1
    • /
    • pp.29-36
    • /
    • 2021
  • The amount of data generated has been growing exponentially, and the complexity of data has been increasing owing to the advancement of information technology (IT). Big data analysts and engineers have therefore been actively conducting research to minimize the analysis targets for faster processing and analysis of big data. Hadoop, which is widely used as a big data platform, provides various processing and analysis functions, including minimization of analysis targets through Hive, which is a subproject of Hadoop. However, Hive uses a vast amount of memory for data deduplication because it is implemented without considering the complexity of data. Therefore, an efficient algorithm has been proposed for data deduplication of complex structures. The performance evaluation results demonstrated that the proposed algorithm reduces the memory usage and data deduplication time by approximately 79% and 0.677%, respectively, compared to Hive. In the future, performance evaluation based on a large number of data nodes is required for a realistic verification of the proposed algorithm.

Construction of Spatial Information Big Data for Urban Thermal Environment Analysis (도시 열환경 분석을 위한 공간정보 빅데이터 구축)

  • Lee, Jun-Hoo;Yoon, Seong-Hwan
    • Journal of the Architectural Institute of Korea Planning & Design
    • /
    • v.36 no.5
    • /
    • pp.53-58
    • /
    • 2020
  • The purpose of this study is to build a database of Spatial information Bigdata of cities using satellite images and spatial information, and to examine the correlations with the surface temperature. Using architectural structure and usage in building information, DEM and Slope topographical information for constructed with 300 × 300 mesh grids for Busan. The satellite image is used to prepare the Normalized Difference Built-up Index (NDBI), Normalized Difference Vegetation Index (NDVI), Bare Soil Index (BI), and Land Surface Temperature (LST). In addition, the building area in the grid was calculated and the building ratio was constructed to build the urban environment DB. In architectural structure, positive correlation was found in masonry and concrete structures. On the terrain, negative correlations were observed between DEM and slope. NDBI and BI were positively correlated, and NDVI was negatively correlated. The higher the Building ratio, the higher the surface temperature. It was found that the urban environment DB could be used as a basic data for urban environment analysis, and it was possible to quantitatively grasp the impact on the architecture and urban environment by adding local meteorological factors. This result is expected to be used as basic data for future urban environment planning and disaster prevention data construction.

Attack Path and Intention Recognition System for detecting APT Attack (APT 공격 탐지를 위한 공격 경로 및 의도 인지 시스템)

  • Kim, Namuk;Eom, Jungho
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.16 no.1
    • /
    • pp.67-78
    • /
    • 2020
  • Typical security solutions such as intrusion detection system are not suitable for detecting advanced persistent attack(APT), because they cannot draw the big picture from trivial events of security solutions. Researches on techniques for detecting multiple stage attacks by analyzing the correlations between security events or alerts are being actively conducted in academic field. However, these studies still use events from existing security system, and there is insufficient research on the structure of the entire security system suitable for advanced persistent attacks. In this paper, we propose an attack path and intention recognition system suitable for multiple stage attacks like advanced persistent attack detection. The proposed system defines the trace format and overall structure of the system that detects APT attacks based on the correlation and behavior analysis, and is designed with a structure of detection system using deep learning and big data technology, etc.