• Title/Summary/Keyword: random fields

Search Result 415, Processing Time 0.022 seconds

A Simultaneous Recognition Technology of Named Entities and Objects for a Dialogue Based Private Secretary Software (대화형 개인 비서 시스템을 위한 하이브리드 방식의 개체명 및 문장목적 동시 인식기술)

  • Lee, ChangSu;Ko, YoungJoong
    • Annual Conference on Human and Language Technology
    • /
    • 2013.10a
    • /
    • pp.18-23
    • /
    • 2013
  • 기존 대화시스템과 달리 대화형 개인 비서 시스템은 사용자에게 정보를 제공하기 위해 앱(APP)을 구동하는 방법을 사용한다. 사용자가 앱을 통해 정보를 얻고자 할 때, 사용자가 필요로 하는 정보를 제공해주기 위해서는 사용자의 목적을 정확하게 인식하는 작업이 필요하다. 그 작업 중 중요한 두 요소는 개체명 인식과 문장목적 인식이다. 문장목적 인식이란, 사용자의 문장을 분석해 하나의 앱에 존재하는 여러 정보 중 사용자가 원하는 정보(문장의 목적)가 무엇인지 찾아주는 인식작업이다. 이러한 인식시스템을 구축하는 방법 중 대표적인 방법은 사전규칙방법과 기계학습방법이다. 사전규칙은 사전정보와 규칙을 적용하는 방법으로, 시간이 지남에 따라 새로운 규칙을 추가해야하는 문제가 있으며, 규칙이 일반화되지 않을 경우 오류가 증가하는 문제가 있다. 또 두 인식작업을 파이프라인 방식으로 적용 할 경우, 개체명 인식단계에서의 오류를 가지고 문장목적 인식단계로 넘어가기 때문에 두 단계에 걸친 성능저하와 속도저하를 초래할 수 있다. 이러한 문제점을 해결하기 위해 우리는 통계기반의 기계학습방법인 Conditional Random Fields(CRF)를 사용한다. 또한 사전정보를 CRF와 결합함으로써, 단독으로 수행하는 CRF방식의 성능을 개선시킨다. 개체명과 문장목적인식의 구조를 분석한 결과, 비슷한 자질을 사용할 수 있다고 판단하여, 두 작업을 동시에 수행하는 방법을 제안한다. 실험결과, 사전규칙방법보다 제안한 방법이 문장단위 2.67% 성능개선을 보였다.

  • PDF

The Sequence Labeling Approach for Text Alignment of Plagiarism Detection

  • Kong, Leilei;Han, Zhongyuan;Qi, Haoliang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.9
    • /
    • pp.4814-4832
    • /
    • 2019
  • Plagiarism detection is increasingly exploiting text alignment. Text alignment involves extracting the plagiarism passages in a pair of the suspicious document and its source document. The heuristics have achieved excellent performance in text alignment. However, the further improvements of the heuristic methods mainly depends more on the experiences of experts, which makes the heuristics lack of the abilities for continuous improvements. To address this problem, machine learning maybe a proper way. Considering the position relations and the context of text segments pairs, we formalize the text alignment task as a problem of sequence labeling, improving the current methods at the model level. Especially, this paper proposes to use the probabilistic graphical model to tag the observed sequence of pairs of text segments. Hence we present the sequence labeling approach for text alignment in plagiarism detection based on Conditional Random Fields. The proposed approach is evaluated on the PAN@CLEF 2012 artificial high obfuscation plagiarism corpus and the simulated paraphrase plagiarism corpus, and compared with the methods achieved the best performance in PAN@CLEF 2012, 2013 and 2014. Experimental results demonstrate that the proposed approach significantly outperforms the state of the art methods.

High Speed Korean Dependency Analysis Using Cascaded Chunking (다단계 구단위화를 이용한 고속 한국어 의존구조 분석)

  • Oh, Jin-Young;Cha, Jeong-Won
    • Journal of the Korea Society for Simulation
    • /
    • v.19 no.1
    • /
    • pp.103-111
    • /
    • 2010
  • Syntactic analysis is an important step in natural language processing. However, we cannot use the syntactic analyzer in Korean for low performance and without robustness. We propose new robust, high speed and high performance Korean syntactic analyzer using CRFs. We treat a parsing problem as a labeling problem. We use a cascaded chunking for Korean parsing. We label syntactic information to each Eojeol at each step using CRFs. CRFs use part-of-speech tag and Eojeol syntactic tag features. Our experimental results using 10-fold cross validation show significant improvement in the robustness, speed and performance of long Korea sentences.

CRF Based Intrusion Detection System using Genetic Search Feature Selection for NSSA

  • Azhagiri M;Rajesh A;Rajesh P;Gowtham Sethupathi M
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.7
    • /
    • pp.131-140
    • /
    • 2023
  • Network security situational awareness systems helps in better managing the security concerns of a network, by monitoring for any anomalies in the network connections and recommending remedial actions upon detecting an attack. An Intrusion Detection System helps in identifying the security concerns of a network, by monitoring for any anomalies in the network connections. We have proposed a CRF based IDS system using genetic search feature selection algorithm for network security situational awareness to detect any anomalies in the network. The conditional random fields being discriminative models are capable of directly modeling the conditional probabilities rather than joint probabilities there by achieving better classification accuracy. The genetic search feature selection algorithm is capable of identifying the optimal subset among the features based on the best population of features associated with the target class. The proposed system, when trained and tested on the bench mark NSL-KDD dataset exhibited higher accuracy in identifying an attack and also classifying the attack category.

Physical Properties of Hardpan in Paddy Fields (논토양 경반의 물리적 특성)

  • Lee, K.S.;Park, J.G.;Cho, S.C.;Noh, K.M.;Chang, Y.C.
    • Journal of Biosystems Engineering
    • /
    • v.32 no.4
    • /
    • pp.207-214
    • /
    • 2007
  • Based on the profiles of cone index with depth, physical properties of hardpan in selected rice fields were measured and analyzed in the study. An error correction algorithm removing a random measurement error from raw CI profile data was introduced in the study. The properties of hardpan included the shape, the thickness and the rice root growing layer. The analysis of physical properties of hardpan in the rice fields showed that the type of hardpan could be classified into 6 categories. The thickness of hardpan ranged from 6 cm up to 41 cm and the average hardness of hardpan was analyzed to be from 1.1 MPa through 3.2 MPa in Cone index.

Comparison on Recent Metastability and Ring-Oscillator TRNGs (최신 준안정성 및 발진기 기반 진 난수 발생기 비교)

  • Shin, Hwasoo;Yoo, Hoyoung
    • Journal of IKEEE
    • /
    • v.24 no.2
    • /
    • pp.543-549
    • /
    • 2020
  • As the importance of security increases in various fields, research on a random number generator (RNG) used for generating an encryption key, has been actively conducted. A high-quality RNG is essential to generate a high-performance encryption key, but the initial pseudo-random number generator (PRNG) has the possibility of predicting the encryption key from the outside even though a large amount of hardware resources are required to generate a sufficiently high-performance random number. Therefore, the demand of high-quality true random number generator (TRNG) generating random number through various noises is increasing. This paper examines and compares the representative TRNG methods based on metastable-based and ring-oscillator-based TRNGs. We compare the methods how the random sources are generated in each TRNG and evaluate its performances using NIST SP 800-22 tests.

A Study on the Rainfall Generation (In Two-dimensional Random Storm Fields) (강우의 모의발생에 관한 연구 (2차원 무작위 호우장에서))

  • Lee, Jea Hyoung;Soun, Jung Ho;Hwang, Man Ha
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.11 no.1
    • /
    • pp.109-116
    • /
    • 1991
  • In recent years, hydrologists have been interested in the radial spectrum and its estimation in two dimensional storm field to construct simulation model of the rainfall. This paper deals with the problem of transformation from the spectrum or isotropic covariance function to two dimensional random field. The extended turning band method for the generation of random field is applied to the problem using the line generation method of one dimensional stochastic process by G.Matheron. Examples of this generation is chosen in the random components of the multidimensional rainfall model suggested by Bras and are given with a comparison between theoretical and sample statistics. In this numerical experiments it is observed that first and second order statistics can be conserved. Also the example of moving storm simulation through Bras model is presented with the appropriate parameters and sample size.

  • PDF

Reliability and risk assessment for rainfall-induced slope failure in spatially variable soils

  • Zhao, Liuyuan;Huang, Yu;Xiong, Min;Ye, Guanbao
    • Geomechanics and Engineering
    • /
    • v.22 no.3
    • /
    • pp.207-217
    • /
    • 2020
  • Slope reliability analysis and risk assessment for spatially variable soils under rainfall infiltration are important subjects but they have not been well addressed. This lack of study may in part be due to the multiple and diverse evaluation indexes and the low computational efficiency of Monte-Carlo simulations. To remedy this, this paper proposes a highly efficient computational method for investigating random field problems for slopes. First, the probability density evolution method (PDEM) is introduced. This method has high computational efficiency and does not need the tens of thousands of numerical simulation samples required by other methods. Second, the influence of rainfall on slope reliability is investigated, where the reliability is calculated from based on the safety factor curves during the rainfall. Finally, the uncertainty of the sliding mass for the slope random field problem is analyzed. Slope failure consequences are considered to be directly correlated with the sliding mass. Calculations showed that the mass that slides is smaller than the potential sliding mass (shallow surface sliding in rainfall). Sliding mass-based risk assessment is both needed and feasible for engineered slope design. The efficient PDEM is recommended for problems requiring lengthy calculations such as random field problems coupled with rainfall infiltration.

The Study on the Mean Residual Life Estimation of Reliability Data under Random Censoring (임의절단 하에서 신뢰성 자료의 평균잔여수명 추정에 대한 연구)

  • Lee, Mi-Sook
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.11 no.6
    • /
    • pp.1997-2003
    • /
    • 2010
  • Mean Residual Life (MRL) function plays a very important role in the area of engineering, medical science, survival studies, social sciences, and many other fields. Specially, in the reliability study of technical systems, the MRL estimation of a component is very important because the sudden stop of a system brings a serious problem. So, many simulation studies of MRL estimation have been done considering various situation variables. In this paper, four estimators of MRL are proposed under random censoring and their performances re compared through bias and Mean Square Error (MSE) by Monte Carlo simulation.

The Effectiveness of the Random-dot E by the Difference of the Illumination and Test Distance (조도와 검사 거리의 차이에 의한 Random-dot E의 영향)

  • Kim, Douk-Hoon
    • Journal of Korean Ophthalmic Optics Society
    • /
    • v.5 no.2
    • /
    • pp.1-4
    • /
    • 2000
  • The test of stereoacuity provides relatively accurate assessment of binocular functior. in the clinical fields. The purpose of this study was to investigate performance on the Random-dot E(RDE) stereotest under the binocular conditions by the difference of the illumination and test distance. The more light illumination increase. The more pass of RDE stereotest increase. On the other hand. On the near distance of test target. All subjects have a pass toe RDE stereatest. But On the far distance of test target. Some subjects have a pass the RDE stereatest. According to the test distance. The more far distance of test target. the less subjects have a pass the RDE stereotest. As a results, The RDE stereotest have effected the test distance and illumination.

  • PDF