• Title/Summary/Keyword: dictionary construction

Search Result 114, Processing Time 0.027 seconds

A Semi-automatic Construction method of a Named Entity Dictionary Based on Wikipedia (위키피디아 기반 개체명 사전 반자동 구축 방법)

  • Song, Yeongkil;Jeong, Seokwon;Kim, Harksoo
    • Journal of KIISE
    • /
    • v.42 no.11
    • /
    • pp.1397-1403
    • /
    • 2015
  • A named entity(NE) dictionary is an important resource for the performance of NE recognition. However, it is not easy to construct a NE dictionary manually since human annotation is time consuming and labor-intensive. To save construction time and reduce human labor, we propose a semi-automatic system for the construction of a NE dictionary. The proposed system constructs a pseudo-document with Wiki-categories per NE class by using an active learning technique. Then, it calculates similarities between Wiki entries and pseudo-documents using the BM25 model, a well-known information retrieval model. Finally, it classifies each Wiki entry into NE classes based on similarities. In experiments with three different types of NE class sets, the proposed system showed high performance(macro-average F1-score of 0.9028 and micro-average F1-score 0.9554).

A Study on Utilization of Wikipedia Contents for Automatic Construction of Linguistic Resources (언어자원 자동 구축을 위한 위키피디아 콘텐츠 활용 방안 연구)

  • Yoo, Cheol-Jung;Kim, Yong;Yun, Bo-Hyun
    • Journal of Digital Convergence
    • /
    • v.13 no.5
    • /
    • pp.187-194
    • /
    • 2015
  • Various linguistic knowledge resources are required in order that machine can understand diverse variation in natural languages. This paper aims to devise an automatic construction method of linguistic resources by reflecting characteristics of online contents toward continuous expansion. Especially we focused to build NE(Named-Entity) dictionary because the applicability of NEs is very high in linguistic analysis processes. Based on the investigation on Korean Wikipedia, we suggested an efficient construction method of NE dictionary using the syntactic patterns and structural features such as metadatas.

Statistical Information of Korean Dictionary to Construct an Enormous Electronic Dictionary (대용량 전자사전 구축을 위한 국어 대사전의 통계 정보)

  • Kim, Cheol-Su;Kim, Yang-Beom
    • The Journal of the Korea Contents Association
    • /
    • v.7 no.6
    • /
    • pp.60-68
    • /
    • 2007
  • There are various application areas of Language information processing such as information retrieval, morphological analysis, spell checker, voice recognition, character recognition, etc. In these language information processing areas, an electronic dictionary is essential. This thesis made researches on basic statistical information on the Korean dictionary and on the construction of electronic dictionary. The targets of analysis were the number of registered word in Korea dictionary, the entry number of registered word in electronic dictionary, the number of used syllables, the number of different syllables, the average length of entry, the distribution of part of speech and the number of used nodes to construct electronic dictionary using Trie, except for words including a archaic word or incomplete syllables. Total entry number of electronic dictionary is 361,980, the number of used syllables is 1,289,659, the average length of entries is 3.56 and the number of different syllables is 2,463. Theses informations would play a beneficial role in constructing an electronic dictionary and in processing Korean information.

Comparing the Usages of Vocabulary by Medias for Disaster Safety Terminology Construction (재난안전 용어사전 구축을 위한 미디어별 어휘 사용 양상 비교)

  • Lee, Jung-Eun;Kim, Tae-Young;Oh, Hyo-Jung
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.7 no.6
    • /
    • pp.229-238
    • /
    • 2018
  • The rapid response of disaster accidents can be archived through the organical involvement of various disaster and safety control agencies. To define the terminology of disaster safety is essential for communication between disaster safety agencies and well as announcement for the public. Also, to efficiently construct a word dictionary of disaster safety terminology, it's necessary to define the priority of the terms. In order to establish direction of word dictionary construction, this paper compares the usage of disaster safety terminology by media: word dictionary, new media, and social media, respectively. Based on the terminology resources collected from each media, we visualized the distribution of terminology according to frequency weights and analyzed co-occurrence patterns. We also classified the types of terminology into four categories and proposed the priority in the construction of disaster safety word dictionary.

Anomaly Detection via Pattern Dictionary Method and Atypicality in Application (패턴사전과 비정형성을 통한 이상치 탐지방법 적용)

  • Sehong Oh;Jongsung Park;Youngsam Yoon
    • Journal of Sensor Science and Technology
    • /
    • v.32 no.6
    • /
    • pp.481-486
    • /
    • 2023
  • Anomaly detection holds paramount significance across diverse fields, encompassing fraud detection, risk mitigation, and sensor evaluation tests. Its pertinence extends notably to the military, particularly within the Warrior Platform, a comprehensive combat equipment system with wearable sensors. Hence, we propose a data-compression-based anomaly detection approach tailored to unlabeled time series and sequence data. This method entailed the construction of two distinctive features, typicality and atypicality, to discern anomalies effectively. The typicality of a test sequence was determined by evaluating the compression efficacy achieved through the pattern dictionary. This dictionary was established based on the frequency of all patterns identified in a training sequence generated for each sensor within Warrior Platform. The resulting typicality served as an anomaly score, facilitating the identification of anomalous data using a predetermined threshold. To improve the performance of the pattern dictionary method, we leveraged atypicality to discern sequences that could undergo compression independently without relying on the pattern dictionary. Consequently, our refined approach integrated both typicality and atypicality, augmenting the effectiveness of the pattern dictionary method. Our proposed method exhibited heightened capability in detecting a spectrum of unpredictable anomalies, fortifying the stability of wearable sensors prevalent in military equipment, including the Army TIGER 4.0 system.

Movie Rating Inference by Construction of Movie Sentiment Sentence using Movie comments and ratings (영화평과 평점을 이용한 감성 문장 구축을 통한 영화 평점 추론)

  • Oh, Yean-Ju;Chae, Soo-Hoan
    • Journal of Internet Computing and Services
    • /
    • v.16 no.2
    • /
    • pp.41-48
    • /
    • 2015
  • On movie review sites, movie ratings are determined by netizens' subjective judgement. This means that inconsistency between ratings and opinions from netizens often occurs. To solve this problem, this paper proposes sentiment sentence sets which affect movie evaluation, and apply sets to comments to infer ratings. Creation of sentiment sentence sets is consisted of two stages, construction of sentiment word dictionary and creation of sentiment sentences for sentiment estimation. Sentiment word dictionary contains sentimental words and its polarities included in reviews. Elements of sentiment sentences are combined with movie related noun and predicate from words sentiment word dictionary. In this study, to make correspondence between polarity of sentiment sentence and sentiment word dictionary, sentiment sentences which have different polarity with sentiment word dictionary are removed. The scores of comments are calculated by applying averages of sentiment sentences elements. The result of experiment shows that sentence scores from sentiment sentence sets are closer to reflect real opinion of comments than ratings by netizens'.

MADE: Morphological Analyzer Development Environment (MADE : 형태소 분석기 개발환경)

  • Shim, Kwang-Seob
    • Journal of Internet Computing and Services
    • /
    • v.8 no.4
    • /
    • pp.159-171
    • /
    • 2007
  • This paper proposes a software tool MADE that is useful to develop a practical Korean morphological analyzer. A morphological analysis is performed by using adjacency conditions provided by a morphological dictionary. This means that developing a morphological analyzer is reduced merely to constructing a morphological dictionary. No programming skill is required in this process, MADE provides with useful functions that facilitate the construction of a dictionary. Once a dictionary is constructed, the morphological analysis engine embedded in MADE may be used as a stand-alone morphological analyzer or be integrated into an application software which requires a Korean morphological analysis module.

  • PDF

Automatic Construction Method of Unknown Word Lexical Dictionary (Unknown Word Lexical Dictionary의 자동 생성 방법)

  • Hwang, Myung-Gwon;Youn, Byung-Su;Jeong, Il-Yong;Kim, Pan-Koo
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2008.05a
    • /
    • pp.3-6
    • /
    • 2008
  • 본 연구는 의미적 정보 검색을 위한 연구 중의 하나로, 현재까지의 의미적 문서 검색에서 큰 걸림돌이었던 사전에 정의되지 않은 단어(Unknown Word)들의 어휘 사전(Lexical Dictionary)을 자동으로 생성하기 위한 것이다. 이를 위해 UW를 기존의 영어 어휘 사전인 워드넷(WordNet)에 정의되지 않은 단어로 간주하고, 웹 문서의 입력을 통하여 UW와 관련된 단어들을 추출하여 의미적 관련 정도를 확률적, 의미적 방법으로 측정한다. 본 논문에서는 UW Lexical Dictionary를 자동으로 구축하기 위한 방법에 대해서만 기술하였고, 정량적이고 객관적인 평가는 포함하지 않고 있다. 하지만 본 연구의 효용성을 확인하기 위한 몇 가지 문서로부터 추출된 결과는 본 연구가 상당히 의미적이며 가치가 높을 것으로 기대되고 있다.

A Development of the Risk Factor Dictionary for the Private Finance Construction Project (민간투자 건설사업 위험요인사전 개발)

  • Kim, Seon-Gyoo
    • Korean Journal of Construction Engineering and Management
    • /
    • v.8 no.5
    • /
    • pp.152-160
    • /
    • 2007
  • Recently, the government is very active to secure the financial resources by inducement of the private investment in fulfilling an extension of the SOC facilities. One of the reasons that the private investors hesitate to put money into the private finance projects, however, is the lack of knowledges about various risks to be possibly incurred during the project execution. This research was performed as one of the preparation works in order for a A engineering company to act as a prime contractor of the project management service in the private finance project, and finally developed the risk factor dictionary as a new concept to satisfy the requirement of a A company and overcome a limit of the existing risk checklists. Although the risk factor dictionary looks like the only simplified table to be the risk factors identified in three dimensions, the impacts and response strategies expressed in narrative and multi-items, and the responsible parties indicated, it has great meanings to get a lot of direct and indirect accomplishments over the simplified table during the development process.

Person Re-identification using Sparse Representation with a Saliency-weighted Dictionary

  • Kim, Miri;Jang, Jinbeum;Paik, Joonki
    • IEIE Transactions on Smart Processing and Computing
    • /
    • v.6 no.4
    • /
    • pp.262-268
    • /
    • 2017
  • Intelligent video surveillance systems have been developed to monitor global areas and find specific target objects using a large-scale database. However, person re-identification presents some challenges, such as pose change and occlusions. To solve the problems, this paper presents an improved person re-identification method using sparse representation and saliency-based dictionary construction. The proposed method consists of three parts: i) feature description based on salient colors and textures for dictionary elements, ii) orthogonal atom selection using cosine similarity to deal with pose and viewpoint change, and iii) measurement of reconstruction error to rank the gallery corresponding a probe object. The proposed method provides good performance, since robust descriptors used as a dictionary atom are generated by weighting some salient features, and dictionary atoms are selected by reducing excessive redundancy causing low accuracy. Therefore, the proposed method can be applied in a large scale-database surveillance system to search for a specific object.