• Title/Summary/Keyword: data matching

Search Result 1,964, Processing Time 0.031 seconds

Noninformative priors for the shape parameter in the generalized Pareto distribution

  • Kang, Sang Gil;Kim, Dal Ho;Lee, Woo Dong
    • Journal of the Korean Data and Information Science Society
    • /
    • v.24 no.1
    • /
    • pp.171-178
    • /
    • 2013
  • In this paper, we develop noninformative priors for the generalized Pareto distribution when the parameter of interest is the shape parameter. We developed the first order and the second order matching priors.We revealed that the second order matching prior does not exist. It turns out that the reference prior satisfies a first order matching criterion, but Jeffrey's prior is not a first order matching prior. Some simulation study is performed and a real example is given.

Optimization Driven MapReduce Framework for Indexing and Retrieval of Big Data

  • Abdalla, Hemn Barzan;Ahmed, Awder Mohammed;Al Sibahee, Mustafa A.
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.5
    • /
    • pp.1886-1908
    • /
    • 2020
  • With the technical advances, the amount of big data is increasing day-by-day such that the traditional software tools face a burden in handling them. Additionally, the presence of the imbalance data in big data is a massive concern to the research industry. In order to assure the effective management of big data and to deal with the imbalanced data, this paper proposes a new indexing algorithm for retrieving big data in the MapReduce framework. In mappers, the data clustering is done based on the Sparse Fuzzy-c-means (Sparse FCM) algorithm. The reducer combines the clusters generated by the mapper and again performs data clustering with the Sparse FCM algorithm. The two-level query matching is performed for determining the requested data. The first level query matching is performed for determining the cluster, and the second level query matching is done for accessing the requested data. The ranking of data is performed using the proposed Monarch chaotic whale optimization algorithm (M-CWOA), which is designed by combining Monarch butterfly optimization (MBO) [22] and chaotic whale optimization algorithm (CWOA) [21]. Here, the Parametric Enabled-Similarity Measure (PESM) is adapted for matching the similarities between two datasets. The proposed M-CWOA outperformed other methods with maximal precision of 0.9237, recall of 0.9371, F1-score of 0.9223, respectively.

Methodology of Spatio-temporal Matching for Constructing an Analysis Database Based on Different Types of Public Data

  • Jung, In taek;Chong, Kyu soo
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.35 no.2
    • /
    • pp.81-90
    • /
    • 2017
  • This study aimed to construct an integrated database using the same spatio-temporal unit by employing various public-data types with different real-time information provision cycles and spatial units. Towards this end, three temporal interpolation methods (piecewise constant interpolation, linear interpolation, nonlinear interpolation) and a spatial matching method by district boundaries was proposed. The case study revealed that the linear interpolation is an excellent method, and the spatial matching method also showed good results. It is hoped that various prediction models and data analysis methods will be developed in the future using different types of data in the analysis database.

Secure Blocking + Secure Matching = Secure Record Linkage

  • Karakasidis, Alexandros;Verykios, Vassilios S.
    • Journal of Computing Science and Engineering
    • /
    • v.5 no.3
    • /
    • pp.223-235
    • /
    • 2011
  • Performing approximate data matching has always been an intriguing problem for both industry and academia. This task becomes even more challenging when the requirement of data privacy rises. In this paper, we propose a novel technique to address the problem of efficient privacy-preserving approximate record linkage. The secure framework we propose consists of two basic components. First, we utilize a secure blocking component based on phonetic algorithms statistically enhanced to improve security. Second, we use a secure matching component where actual approximate matching is performed using a novel private approach of the Levenshtein Distance algorithm. Our goal is to combine the speed of private blocking with the increased accuracy of approximate secure matching.

Noninformative Priors for the Ratio of the Scale Parameters in the Inverted Exponential Distributions

  • Kang, Sang Gil;Kim, Dal Ho;Lee, Woo Dong
    • Communications for Statistical Applications and Methods
    • /
    • v.20 no.5
    • /
    • pp.387-394
    • /
    • 2013
  • In this paper, we develop the noninformative priors for the ratio of the scale parameters in the inverted exponential distributions. The first and second order matching priors, the reference prior and Jeffreys prior are developed. It turns out that the second order matching prior matches the alternative coverage probabilities, is a cumulative distribution function matching prior and is a highest posterior density matching prior. In addition, the reference prior and Jeffreys' prior are the second order matching prior. We show that the proposed reference prior matches the target coverage probabilities in a frequentist sense through a simulation study as well as provide an example based on real data is given.

Performance analysis of transfer alignment for velocity & angle matching and angular rate & acceleration matching (속도 및 각정합과 각속도 및 가속도정합에 대한 전달정렬의 특성 분석)

  • 양철관;심덕선
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 1997.10a
    • /
    • pp.1860-1863
    • /
    • 1997
  • Transfer alignement is the process of initializing attitude of slave INS using the data of master INS. This paper presents the performance analysis of transfer alignment at sea using convariance analysis method. Velocity & angle matching and angular rate & acceleration matching are used for analysis, and the performance of two matching methods are compared. We propose a new method for angular rate & acceleration matching. Under the assumption of accurate modeling of ship flexure, the performance of transfer alignment time and accuray is improved very much for the new method.

  • PDF

DEVELOPING NONINFORMATIVE PRIORS FOR THE FAMILIAL DATA

  • Heo, Jung-Eun;Kim, Yeong-Hwa
    • Journal of the Korean Statistical Society
    • /
    • v.36 no.1
    • /
    • pp.77-91
    • /
    • 2007
  • This paper considers development of noninformative priors for the familial data when the families have equal number of offspring. Several noninformative priors including the widely used Jeffreys' prior as well as the different reference priors are derived. Also, a simultaneously-marginally-probability-matching prior is considered and probability matching priors are derived when the parameter of interest is inter- or intra-class correlation coefficient. The simulation study implemented by Gibbs sampler shows that two-group reference prior is slightly edge over the others in terms of coverage probability.

An efficient matching mechanism for real-time sensor data dissemination (실시간 센서 데이터 배포를 위한 효율적 매칭)

  • Seok, Bo-Hyun;Lee, Pill-Woo;Huh, Eui-Nam
    • Journal of Internet Computing and Services
    • /
    • v.9 no.1
    • /
    • pp.79-90
    • /
    • 2008
  • In the ubiquitous environment sensor network technologies have advanced for collecting information of the environment. With the rapid growth of sensor network technology, it is necessary and important to share the collected sensor data with a large base of diverse users. In order to provide dissemination of sensor data, we design an information dissemination system using an independent disseminator between provider and consumer. This paper describes how we designed the information dissemination system using one of the possible dissemination patterns for sensor networks, and an efficient matching algorithm called CGIM (Classed Grouping Index Matching) which employs a dynamic re-grouping scheme.

  • PDF

Rail Profile Matching Method using ICP Algorithm (ICP 알고리즘을 이용한 레일 프로파일 매칭 기법)

  • Yu, Young-Ki;Koo, Ja-Myung;Oh, Min-Soo;Yang, Il-Dong
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.65 no.5
    • /
    • pp.888-894
    • /
    • 2016
  • In this paper, we describe a method for precisely measuring the abrasion of the railway using an image processing technique. To calculate the wear of the rails, we provided a method for accurately matching the standard rail profile data and the profile data acquired by the rail inspection vehicle. After the lens distortion correction and the perspective transformation of the measured profile data, we used ICP Algorithm for accurate profile data matching with the reference profile extracted from the standard rail drawing. We constructed the prototype of the Rail Profile Measurement System for High-speed Railway and the experimental result on the three type of the standard rail used in Korea showed the excellent profile matching accuracy within 0.1mm.

Determination of Aquifer Parameters by the Improved Slope-Matching Method (개선된 SM(Slope-Matching) 방법에 의한 대수층의 특성변수 결정)

  • 김민환;오종민;전일권
    • Journal of Soil and Groundwater Environment
    • /
    • v.6 no.4
    • /
    • pp.73-81
    • /
    • 2001
  • With the development of industry and standard of living, the quantity of groundwater consumption has been increasing. For the analysis of groundwater, to determine the hydraulic parameters of aquifer is very important. Various numerical methods have been developed to solve aquifer tests and eliminate the subjectivity of traditional graphical type curve methods. The slope-matching method, which matches the slope of the Theis type curve to the slope of the field data, can be used to numerically solve pump tests for both leaky and nonleaky aquifers. A FORTRAN program on based slope-matching method was developed to obtain the transmissivity, storage coefficient, and leaky factor from pumping test data automatically. Results derived from published data show that the improved slope-matching method gives parameters close to the ones derived by the slope-matching method.

  • PDF