• Title/Summary/Keyword: Stemming algorithms

Search Result 7, Processing Time 0.025 seconds

Information Retrieval Systems: Between Morphological Analyzers and Systemming Algorithms

  • Mohamed, Afaf Abdel Rhman;Ouni, Chafika;Eljack, Sarah Mustafa;Alfayez, Fayez
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.3
    • /
    • pp.375-381
    • /
    • 2022
  • The main objective of an Information Retrieval System (IRS) is to obtain suitable information within a reasonable time to satisfy a user need. To achieve this purpose, an IRS should have a good indexing system that is based on natural language processing.In this context, we focus on the available Arabic language processing techniques for an IRS with the goal of contributing to an improvement in the performance. Our contribution consists of integrating morphological analysis into an IRS in order to compare the impact of morphological analysis with that of stemming algorithms.

Comparative Study of Various Persian Stemmers in the Field of Information Retrieval

  • Moghadam, Fatemeh Momenipour;Keyvanpour, MohammadReza
    • Journal of Information Processing Systems
    • /
    • v.11 no.3
    • /
    • pp.450-464
    • /
    • 2015
  • In linguistics, stemming is the operation of reducing words to their more general form, which is called the 'stem'. Stemming is an important step in information retrieval systems, natural language processing, and text mining. Information retrieval systems are evaluated by metrics like precision and recall and the fundamental superiority of an information retrieval system over another one is measured by them. Stemmers decrease the indexed file, increase the speed of information retrieval systems, and improve the performance of these systems by boosting precision and recall. There are few Persian stemmers and most of them work based on morphological rules. In this paper we carefully study Persian stemmers, which are classified into three main classes: structural stemmers, lookup table stemmers, and statistical stemmers. We describe the algorithms of each class carefully and present the weaknesses and strengths of each Persian stemmer. We also propose some metrics to compare and evaluate each stemmer by them.

Automated Essay Grading: An Application For Historical Malay Text

  • Syed Mustapha, S.M.F.D;Idris, N.
    • Proceedings of the Korea Inteligent Information System Society Conference
    • /
    • 2001.01a
    • /
    • pp.237-245
    • /
    • 2001
  • Automated essay grading has been proposed for over thirty years. Only recently have practical implementations been constructed and tested. This paper investigated the role of the nearest-neighbour algorithm within the information retrieval as a way of grading the essay automatically called Automated Essay Grading System. It intended to offer teachers an individualized assistance in grading the student\`s essay. The system involved several processes, which are the indexing, the structuring of the model answer and the grade processing. The indexing process comprised the document indexing and query processing which are mainly used for representing the documents and the query. Structuring the model answer is actually preparing the marking scheme and the grade processing is the process of assessing the essay. To test the effectiveness of the developed algorithms, the algorithms are tested against the History text in Malay. The result showed that th information retrieval and the nearest-neighbour algorithm are practical combination that offer acceptable performance for grading the essay.

  • PDF

Projecting the Potential Distribution of Abies koreana in Korea Under the Climate Change Based on RCP Scenarios (RCP 기후변화 시나리오에 따른 우리나라 구상나무 잠재 분포 변화 예측)

  • Koo, Kyung Ah;Kim, Jaeuk;Kong, Woo-seok;Jung, Huicheul;Kim, Geunhan
    • Journal of the Korean Society of Environmental Restoration Technology
    • /
    • v.19 no.6
    • /
    • pp.19-30
    • /
    • 2016
  • The projection of climate-related range shift is critical information for conservation planning of Korean fir (Abies koreana E. H. Wilson). We first modeled the distribution of Korean fir under current climate condition using five single-model species distribution models (SDMs) and the pre-evaluation weighted ensemble method and then predicted the distributions under future climate conditions projected with HadGEM2-AO under four $CO_2$ emission scenarios, the Representative Concentration Pathways (RCP) 2.6, 4.5, 6.0 and 8.5. We also investigated the predictive uncertainty stemming from five individual algorithms and four $CO_2$ emission scenarios for better interpretation of SDM projections. Five individual algorithms were Generalized linear model (GLM), Generalized additive model (GAM), Multivariate adaptive regression splines (MARS), Generalized boosted model (GBM) and Random forest (RF). The results showed high variations of model performances among individual SDMs and the wide range of diverging predictions of future distributions of Korean fir in response to RCPs. The ensemble model presented the highest predictive accuracy (TSS = 0.97, AUC = 0.99) and predicted that the climate habitat suitability of Korean fir would increase under climate changes. Accordingly, the fir distribution could expand under future climate conditions. Increasing precipitation may account for increases in the distribution of Korean fir. Increasing precipitation compensates the negative effects of increasing temperature. However, the future distribution of Korean fir is also affected by other ecological processes, such as interactions with co-existing species, adaptation and dispersal limitation, and other environmental factors, such as extreme weather events and land-use changes. Therefore, we need further ecological research and to develop mechanistic and process-based distribution models for improving the predictive accuracy.

Meta-heuristic optimization algorithms for prediction of fly-rock in the blasting operation of open-pit mines

  • Mahmoodzadeh, Arsalan;Nejati, Hamid Reza;Mohammadi, Mokhtar;Ibrahim, Hawkar Hashim;Rashidi, Shima;Mohammed, Adil Hussein
    • Geomechanics and Engineering
    • /
    • v.30 no.6
    • /
    • pp.489-502
    • /
    • 2022
  • In this study, a Gaussian process regression (GPR) model as well as six GPR-based metaheuristic optimization models, including GPR-PSO, GPR-GWO, GPR-MVO, GPR-MFO, GPR-SCA, and GPR-SSO, were developed to predict fly-rock distance in the blasting operation of open pit mines. These models included GPR-SCA, GPR-SSO, GPR-MVO, and GPR. In the models that were obtained from the Soungun copper mine in Iran, a total of 300 datasets were used. These datasets included six input parameters and one output parameter (fly-rock). In order to conduct the assessment of the prediction outcomes, many statistical evaluation indices were used. In the end, it was determined that the performance prediction of the ML models to predict the fly-rock from high to low is GPR-PSO, GPR-GWO, GPR-MVO, GPR-MFO, GPR-SCA, GPR-SSO, and GPR with ranking scores of 66, 60, 54, 46, 43, 38, and 30 (for 5-fold method), respectively. These scores correspond in conclusion, the GPR-PSO model generated the most accurate findings, hence it was suggested that this model be used to forecast the fly-rock. In addition, the mutual information test, also known as MIT, was used in order to investigate the influence that each input parameter had on the fly-rock. In the end, it was determined that the stemming (T) parameter was the most effective of all the parameters on the fly-rock.

Development of a Single Allocation Hub Network Design Model with Transportation Economies of Scale (수송 규모의 경제 효과를 고려한 단일 할당 허브 네트워크 설계 모형의 개발)

  • Kim, Dong Kyu;Park, Chang Ho;Lee, Jin Su
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.26 no.6D
    • /
    • pp.917-926
    • /
    • 2006
  • Transportation Economies of scale are the essential properties of hub networks. One critical property of the hub network design problem is to quantify cost savings which stem from economies of scale, the costs of operating hub facilities and opportunity costs associated with delays stemming from consolidation of traffic flows. Due to the NP-complete property of the hub location problem, however, most previous researchers have focused on the development of heuristic algorithms for approximate solutions. The purpose of this paper is to develop a hub network design model considering transportation economies of scale from the consolidation of traffic flows. The model is designed to consider the uniqueness of hub networks and to determine several cost components. The heuristic algorithms for the developed model are suggested and the results of the model are compared with recently published studies using real data. Results of the analysis show that the proposed model reflects transportation economies of scale due to consolidation of flows. This study can form not only the theoretical basis of an effective and rational hub network design but contribute to the assessment of existing and planned logistics systems.

Latent topics-based product reputation mining (잠재 토픽 기반의 제품 평판 마이닝)

  • Park, Sang-Min;On, Byung-Won
    • Journal of Intelligence and Information Systems
    • /
    • v.23 no.2
    • /
    • pp.39-70
    • /
    • 2017
  • Data-drive analytics techniques have been recently applied to public surveys. Instead of simply gathering survey results or expert opinions to research the preference for a recently launched product, enterprises need a way to collect and analyze various types of online data and then accurately figure out customer preferences. In the main concept of existing data-based survey methods, the sentiment lexicon for a particular domain is first constructed by domain experts who usually judge the positive, neutral, or negative meanings of the frequently used words from the collected text documents. In order to research the preference for a particular product, the existing approach collects (1) review posts, which are related to the product, from several product review web sites; (2) extracts sentences (or phrases) in the collection after the pre-processing step such as stemming and removal of stop words is performed; (3) classifies the polarity (either positive or negative sense) of each sentence (or phrase) based on the sentiment lexicon; and (4) estimates the positive and negative ratios of the product by dividing the total numbers of the positive and negative sentences (or phrases) by the total number of the sentences (or phrases) in the collection. Furthermore, the existing approach automatically finds important sentences (or phrases) including the positive and negative meaning to/against the product. As a motivated example, given a product like Sonata made by Hyundai Motors, customers often want to see the summary note including what positive points are in the 'car design' aspect as well as what negative points are in thesame aspect. They also want to gain more useful information regarding other aspects such as 'car quality', 'car performance', and 'car service.' Such an information will enable customers to make good choice when they attempt to purchase brand-new vehicles. In addition, automobile makers will be able to figure out the preference and positive/negative points for new models on market. In the near future, the weak points of the models will be improved by the sentiment analysis. For this, the existing approach computes the sentiment score of each sentence (or phrase) and then selects top-k sentences (or phrases) with the highest positive and negative scores. However, the existing approach has several shortcomings and is limited to apply to real applications. The main disadvantages of the existing approach is as follows: (1) The main aspects (e.g., car design, quality, performance, and service) to a product (e.g., Hyundai Sonata) are not considered. Through the sentiment analysis without considering aspects, as a result, the summary note including the positive and negative ratios of the product and top-k sentences (or phrases) with the highest sentiment scores in the entire corpus is just reported to customers and car makers. This approach is not enough and main aspects of the target product need to be considered in the sentiment analysis. (2) In general, since the same word has different meanings across different domains, the sentiment lexicon which is proper to each domain needs to be constructed. The efficient way to construct the sentiment lexicon per domain is required because the sentiment lexicon construction is labor intensive and time consuming. To address the above problems, in this article, we propose a novel product reputation mining algorithm that (1) extracts topics hidden in review documents written by customers; (2) mines main aspects based on the extracted topics; (3) measures the positive and negative ratios of the product using the aspects; and (4) presents the digest in which a few important sentences with the positive and negative meanings are listed in each aspect. Unlike the existing approach, using hidden topics makes experts construct the sentimental lexicon easily and quickly. Furthermore, reinforcing topic semantics, we can improve the accuracy of the product reputation mining algorithms more largely than that of the existing approach. In the experiments, we collected large review documents to the domestic vehicles such as K5, SM5, and Avante; measured the positive and negative ratios of the three cars; showed top-k positive and negative summaries per aspect; and conducted statistical analysis. Our experimental results clearly show the effectiveness of the proposed method, compared with the existing method.