• 제목/요약/키워드: limited data

검색결과 6,503건 처리시간 0.039초

고해상도 수치예측자료 생산을 위한 경도-역거리 제곱법(GIDS) 기반의 공간 규모 상세화 기법 활용 (Implementation of Spatial Downscaling Method Based on Gradient and Inverse Distance Squared (GIDS) for High-Resolution Numerical Weather Prediction Data)

  • 양아련;오수빈;김주완;이승우;김춘지;박수현
    • 대기
    • /
    • 제31권2호
    • /
    • pp.185-198
    • /
    • 2021
  • In this study, we examined a spatial downscaling method based on Gradient and Inverse Distance Squared (GIDS) weighting to produce high-resolution grid data from a numerical weather prediction model over Korean Peninsula with complex terrain. The GIDS is a simple and effective geostatistical downscaling method using horizontal distance gradients and an elevation. The predicted meteorological variables (e.g., temperature and 3-hr accumulated rainfall amount) from the Limited-area ENsemble prediction System (LENS; horizontal grid spacing of 3 km) are used for the GIDS to produce a higher horizontal resolution (1.5 km) data set. The obtained results were compared to those from the bilinear interpolation. The GIDS effectively produced high-resolution gridded data for temperature with the continuous spatial distribution and high dependence on topography. The results showed a better agreement with the observation by increasing a searching radius from 10 to 30 km. However, the GIDS showed relatively lower performance for the precipitation variable. Although the GIDS has a significant efficiency in producing a higher resolution gridded temperature data, it requires further study to be applied for rainfall events.

Cable damage identification of cable-stayed bridge using multi-layer perceptron and graph neural network

  • Pham, Van-Thanh;Jang, Yun;Park, Jong-Woong;Kim, Dong-Joo;Kim, Seung-Eock
    • Steel and Composite Structures
    • /
    • 제44권2호
    • /
    • pp.241-254
    • /
    • 2022
  • The cables in a cable-stayed bridge are critical load-carrying parts. The potential damage to cables should be identified early to prevent disasters. In this study, an efficient deep learning model is proposed for the damage identification of cables using both a multi-layer perceptron (MLP) and a graph neural network (GNN). Datasets are first generated using the practical advanced analysis program (PAAP), which is a robust program for modeling and analyzing bridge structures with low computational costs. The model based on the MLP and GNN can capture complex nonlinear correlations between the vibration characteristics in the input data and the cable system damage in the output data. Multiple hidden layers with an activation function are used in the MLP to expand the original input vector of the limited measurement data to obtain a complete output data vector that preserves sufficient information for constructing the graph in the GNN. Using the gated recurrent unit and set2set model, the GNN maps the formed graph feature to the output cable damage through several updating times and provides the damage results to both the classification and regression outputs. The model is fine-tuned with the original input data using Adam optimization for the final objective function. A case study of an actual cable-stayed bridge was considered to evaluate the model performance. The results demonstrate that the proposed model provides high accuracy (over 90%) in classification and satisfactory correlation coefficients (over 0.98) in regression and is a robust approach to obtain effective identification results with a limited quantity of input data.

Spaceborne High Speed Data Link Design for Multi-Mode SAR Image Data Transmission

  • Kwag, Young-Kil
    • Journal of electromagnetic engineering and science
    • /
    • 제2권1호
    • /
    • pp.39-44
    • /
    • 2002
  • A high speed data link capability is one of the critical factors in determining the performance of the spaceborne SAR system with high resolution because of the strict requirement far the real-time data transmission of the massive SAR data in a limited time of mission. In this paper, based on the data lint model characterized by the spaceborne small SAR system, the high rate multi-channel data link module is designed including link storage, link processor, transmitter, and wide-angle antenna. The design results are presented with the performance analysis on the data link budget as well as the multi-mode data rate in association with the SAR imaging mode of operation from high resolution to the wide swath.

Domain Adaptation for Opinion Classification: A Self-Training Approach

  • Yu, Ning
    • Journal of Information Science Theory and Practice
    • /
    • 제1권1호
    • /
    • pp.10-26
    • /
    • 2013
  • Domain transfer is a widely recognized problem for machine learning algorithms because models built upon one data domain generally do not perform well in another data domain. This is especially a challenge for tasks such as opinion classification, which often has to deal with insufficient quantities of labeled data. This study investigates the feasibility of self-training in dealing with the domain transfer problem in opinion classification via leveraging labeled data in non-target data domain(s) and unlabeled data in the target-domain. Specifically, self-training is evaluated for effectiveness in sparse data situations and feasibility for domain adaptation in opinion classification. Three types of Web content are tested: edited news articles, semi-structured movie reviews, and the informal and unstructured content of the blogosphere. Findings of this study suggest that, when there are limited labeled data, self-training is a promising approach for opinion classification, although the contributions vary across data domains. Significant improvement was demonstrated for the most challenging data domain-the blogosphere-when a domain transfer-based self-training strategy was implemented.

Pagoda Data Management and Metadata Requirements for Libraries in Myanmar

  • Tin Tin Pipe;Kulthida Tuamsuk
    • Journal of Information Science Theory and Practice
    • /
    • 제11권3호
    • /
    • pp.79-91
    • /
    • 2023
  • The storage of data documentation for Myanmar pagodas has various issues, and its retrieval method causes problems for users and libraries. This study utilized a mixed-methods approach, combining qualitative and quantitative methods to investigate pagoda data management in Myanmar libraries. The study aims to achieve the following objectives: to study the library collection management of pagodas in Myanmar, to investigate the management of pagoda data in Myanmar libraries, and to identify the pagoda data requirements for metadata development from the library professional perspective. The study findings revealed several challenges facing librarians and library users in accessing and managing Myanmar pagoda data, including limited stocks and retrieval tools, difficulty in accessing all available data online, and a lack of a centralized database or repository for storing and retrieving pagoda data. The study recommends the establishment of metadata criteria for managing a set of pagoda data and improving access to technology to address these challenges.

XML 레이블링을 이용한 XML 조각 스트림에 대한 질의 처리 기법 (A Query Processing Technique for XML Fragment Stream using XML Labeling)

  • 이상욱;김진;강현철
    • 한국정보과학회논문지:데이타베이스
    • /
    • 제35권1호
    • /
    • pp.67-83
    • /
    • 2008
  • 유비쿼터스 컴퓨팅의 실현을 위해서는 이동 단말기의 자원 및 컴퓨팅 파워의 효율적 사용이 필수적이다. 특히, 이동 단말기에 내장된 소프트웨어의 수행에 있어 메모리 효율성 에너지 효율성, 그리고 처리 효율성이 요구된다. 본 논문은 자원이 제약되어 있는 이동 단말기에서의 XML 데이타에 대한 질의 처리에 관한 것이다. 메모리 용량이 크지 않은 단말기의 경우 대량의 XML 데이타에 대한 질의 처리를 수행하기 위해서는 XML 스트림 질의 처리 기술이 활용되어야 한다. 최근에 제시된 XFrag는 홀-필러 모델을 이용하여 XML 데이타를 XML 조각으로 분할하여 스트림으로 전송하고 처리할 수 있는 기법이다. 이는 메모리가 부족한 이동 단말기에서 조각 스트림으로부터 XML 데이타를 재구성하지 않고 질의 처리를 가능하게 한다. 그러나 홀-필러 모델을 사용할 경우 홀과 필러에 대한 부가적인 정보를 저장해야 하므로 메모리 효율성이 높지 못하다. 본 논문에서는 XML 데이타의 구조 정보를 표현하는 XML 레이블링 기법을 이용하여 XML 데이타를 조각으로 분할하여 처리하는 새로운 기법 XFLab을 제시한다. 구현 및 성능 실험 결과 XFLab이 XFrag보다 메모리 사용량과 처리 시간 양면 모두에서 우수한 것으로 나타났다.

표면 결함 검출을 위한 데이터 확장 및 성능분석 (Performance Analysis of Data Augmentation for Surface Defects Detection)

  • 김준봉;서기성
    • 전기학회논문지
    • /
    • 제67권5호
    • /
    • pp.669-674
    • /
    • 2018
  • Data augmentation is an efficient way to reduce overfitting on models and to improve a performance supplementing extra data for training. It is more important in deep learning based industrial machine vision. Because deep learning requires huge scale of learning data to learn a model, but acquisition of data can be limited in most of industrial applications. A very generic method for augmenting image data is to perform geometric transformations, such as cropping, rotating, translating and adjusting brightness of the image. The effectiveness of data augmentation in image classification has been reported, but it is rare in defect inspections. We explore and compare various basic augmenting operations for the metal surface defects. The experiments were executed for various types of defects and different CNN networks and analysed for performance improvements by the data augmentations.

Development of the Unified Database Design Methodology for Big Data Applications - based on MongoDB -

  • Lee, Junho;Joo, Kyungsoo
    • 한국컴퓨터정보학회논문지
    • /
    • 제23권3호
    • /
    • pp.41-48
    • /
    • 2018
  • The recent sudden increase of big data has characteristics such as continuous generation of data, large amount, and unstructured format. The existing relational database technologies are inadequate to handle such big data due to the limited processing speed and the significant storage expansion cost. Current implemented solutions are mainly based on relational database that are no longer adapted to these data volume. NoSQL solutions allow us to consider new approaches for data warehousing, especially from the multidimensional data management point of view. In this paper, we develop and propose the integrated design methodology based on MongoDB for big data applications. The proposed methodology is more scalable than the existing methodology, so it is easy to handle big data.

An Efficient Information Retrieval System for Unstructured Data Using Inverted Index

  • Abdullah Iftikhar;Muhammad Irfan Khan;Kulsoom Iftikhar
    • International Journal of Computer Science & Network Security
    • /
    • 제24권7호
    • /
    • pp.31-44
    • /
    • 2024
  • The inverted index is combination of the keywords and posting lists associated for indexing of document. In modern age excessive use of technology has increased data volume at a very high rate. Big data is great concern of researchers. An efficient Document indexing in big data has become a major challenge for researchers. All organizations and web engines have limited number of resources such as space and storage which is very crucial in term of data management of information retrieval system. Information retrieval system need to very efficient. Inverted indexing technique is introduced in this research to minimize the delay in retrieval of data in information retrieval system. Inverted index is illustrated and then its issues are discussed and resolve by implementing the scalable inverted index. Then existing algorithm of inverted compared with the naïve inverted index. The Interval list of inverted indexes stores on primary storage except of auxiliary memory. In this research an efficient architecture of information retrieval system is proposed particularly for unstructured data which don't have a predefined structure format and data volume.

Dual Generalized Maximum Entropy Estimation for Panel Data Regression Models

  • Lee, Jaejun;Cheon, Sooyoung
    • Communications for Statistical Applications and Methods
    • /
    • 제21권5호
    • /
    • pp.395-409
    • /
    • 2014
  • Data limited, partial, or incomplete are known as an ill-posed problem. If the data with ill-posed problems are analyzed by traditional statistical methods, the results obviously are not reliable and lead to erroneous interpretations. To overcome these problems, we propose a dual generalized maximum entropy (dual GME) estimator for panel data regression models based on an unconstrained dual Lagrange multiplier method. Monte Carlo simulations for panel data regression models with exogeneity, endogeneity, or/and collinearity show that the dual GME estimator outperforms several other estimators such as using least squares and instruments even in small samples. We believe that our dual GME procedure developed for the panel data regression framework will be useful to analyze ill-posed and endogenous data sets.