• Title/Summary/Keyword: Log Clustering

Search Result 73, Processing Time 0.025 seconds

Fuzzy Training Based on Segmentation Using Spatial Region Growing

  • Lee Sang-Hoon
    • Korean Journal of Remote Sensing
    • /
    • v.20 no.5
    • /
    • pp.353-359
    • /
    • 2004
  • This study proposes an approach to unsupervisedly estimate the number of classes and the parameters of defining the classes in order to train the classifier. In the proposed method, the image is segmented using a spatial region growing based on hierarchical clustering, and fuzzy training is then employed to find the sample classes that well represent the ground truth. For cluster validation, this approach iteratively estimates the class-parameters in the fuzzy training for the sample classes and continuously computes the log-likelihood ratio of two consecutive class-numbers. The maximum ratio rule is applied to determine the optimal number of classes. The experimental results show that the new scheme proposed in this study could be used to select the regions with different characteristics existed on the scene of observed image as an alternative of field survey that is so expensive.

Designing mobile personal assistant agent based on users' experience and their position information (위치정보 및 사용자 경험을 반영하는 모바일 PA에이전트의 설계)

  • Kang, Shin-Bong;Noh, Sang-Uk
    • Journal of Internet Computing and Services
    • /
    • v.12 no.1
    • /
    • pp.99-110
    • /
    • 2011
  • Mobile environments rapidly changing and digital convergence widely employed, mobile devices including smart phones have been playing a critical role that changes users' lifestyle in the areas of entertainments, businesses and information services. The various services using mobile devices are developing to meet the personal needs of users in the mobile environments. Especially, an LBS (Location-Based Service) is combined with other services and contents such as augmented reality, mobile SNS (Social Network Service), games, and searching, which can provide convenient and useful services to mobile users. In this paper, we design and implement the prototype of mobile personal assistant (PA) agents. Our personal assistant agent helps users do some tasks by hiding the complexity of difficult tasks, performing tasks on behalf of the users, and reflecting the preferences of users. To identify user's preferences and provide personalized services, clustering and classification algorithms of data mining are applied. The clusters of the log data using clustering algorithms are made by measuring the dissimilarity between two objects based on usage patterns. The classification algorithms produce user profiles within each cluster, which make it possible for PA agents to provide users with personalized services and contents. In the experiment, we measured the classification accuracy of user model clustered using clustering algorithms. It turned out that the classification accuracy using our method was increased by 17.42%, compared with that using other clustering algorithms.

Anomaly Detection Analysis using Repository based on Inverted Index (역방향 인덱스 기반의 저장소를 이용한 이상 탐지 분석)

  • Park, Jumi;Cho, Weduke;Kim, Kangseok
    • Journal of KIISE
    • /
    • v.45 no.3
    • /
    • pp.294-302
    • /
    • 2018
  • With the emergence of the new service industry due to the development of information and communication technology, cyber space risks such as personal information infringement and industrial confidentiality leakage have diversified, and the security problem has emerged as a critical issue. In this paper, we propose a behavior-based anomaly detection method that is suitable for real-time and large-volume data analysis technology. We show that the proposed detection method is superior to existing signature security countermeasures that are based on large-capacity user log data according to in-company personal information abuse and internal information leakage. As the proposed behavior-based anomaly detection method requires a technique for processing large amounts of data, a real-time search engine is used, called Elasticsearch, which is based on an inverted index. In addition, statistical based frequency analysis and preprocessing were performed for data analysis, and the DBSCAN algorithm, which is a density based clustering method, was applied to classify abnormal data with an example for easy analysis through visualization. Unlike the existing anomaly detection system, the proposed behavior-based anomaly detection technique is promising as it enables anomaly detection analysis without the need to set the threshold value separately, and was proposed from a statistical perspective.

The Effect of Small-World Structure in Team Processes on Team Performance (팀 프로세스의 작은 세상 구조가 팀 성과에 미치는 영향)

  • Seo, Il-Jung
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.3
    • /
    • pp.539-547
    • /
    • 2019
  • This study investigated the effect of small-world structure in team processes on team performance. I discussed the theoretical relationship between small-world structure in team processes and team performance and analyzed the relationship using pass data of soccer teams. I constructed the 128 pass networks from the pass data of the 2014 FIFA World Cup and then measured the structural features indicating small-world structure of the networks. Correlation analysis and regression analysis were performed in order to examine the strength and direction of the relationship. According to the results, the clustering has an exponential relationship with team performance and the connectivity has a log-function relationship with team performance. Finally, I found the positive effect of small-world structure in team processes on team performance. Through theoretical discussion and empirical analysis, this study found that small-world structure in team processes increase team performance by facilitating task coordination and collaboration between team members.

Decision Support System to Detect Unauthorized Access in Smart Work Environment (스마트워크 환경에서 이상접속탐지를 위한 의사결정지원 시스템 연구)

  • Lee, Jae-Ho;Lee, Dong-Hoon;Kim, Huy-Kang
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.22 no.4
    • /
    • pp.797-808
    • /
    • 2012
  • In smart work environment, a company provides employees a flexible work environment for tele-working using mobile phone or portable devices. On the other hand, such environment are exposed to the risks which the attacker can intrude into computer systems or leak personal information of smart-workers' and gain a company's sensitive information. To reduce these risks, the security administrator needs to analyze the usage patterns of employees and detect abnormal behaviors by monitoring VPN(Virtual Private Network) access log. This paper proposes a decision support system that can notify the status by using visualization and similarity measure through clustering analysis. On average, 88.7% of abnormal event can be detected by this proposed method. With this proposed system, the security administrator can detect abnormal behaviors of the employees and prevent account theft.

A streamlined pipeline based on HmmUFOtu for microbial community profiling using 16S rRNA amplicon sequencing

  • Hyeonwoo Kim;Jiwon Kim;Ji Won Cho;Kwang-Sung Ahn;Dong-Il Park;Sangsoo Kim
    • Genomics & Informatics
    • /
    • v.21 no.3
    • /
    • pp.40.1-40.11
    • /
    • 2023
  • Microbial community profiling using 16S rRNA amplicon sequencing allows for taxonomic characterization of diverse microorganisms. While amplicon sequence variant (ASV) methods are increasingly favored for their fine-grained resolution of sequence variants, they often discard substantial portions of sequencing reads during quality control, particularly in datasets with large number samples. We present a streamlined pipeline that integrates FastP for read trimming, HmmUFOtu for operational taxonomic units (OTU) clustering, Vsearch for chimera checking, and Kraken2 for taxonomic assignment. To assess the pipeline's performance, we reprocessed two published stool datasets of normal Korean populations: one with 890 and the other with 1,462 independent samples. In the first dataset, HmmUFOtu retained 93.2% of over 104 million read pairs after quality trimming, discarding chimeric or unclassifiable reads, while DADA2, a commonly used ASV method, retained only 44.6% of the reads. Nonetheless, both methods yielded qualitatively similar β-diversity plots. For the second dataset, HmmUFOtu retained 89.2% of read pairs, while DADA2 retained a mere 18.4% of the reads. HmmUFOtu, being a closed-reference clustering method, facilitates merging separately processed datasets, with shared OTUs between the two datasets exhibiting a correlation coefficient of 0.92 in total abundance (log scale). While the first two dimensions of the β-diversity plot exhibited a cohesive mixture of the two datasets, the third dimension revealed the presence of a batch effect. Our comparative evaluation of ASV and OTU methods within this streamlined pipeline provides valuable insights into their performance when processing large-scale microbial 16S rRNA amplicon sequencing data. The strengths of HmmUFOtu and its potential for dataset merging are highlighted.

A Statistical Image Segmentation Method in the Hierarchical Image Structure (계층적 영상구조에서 통계적 방법에 의한 영상분할)

  • 최성진
    • Journal of Broadcast Engineering
    • /
    • v.1 no.2
    • /
    • pp.165-175
    • /
    • 1996
  • In this paper, the image segmentation method based on the hierarchical pyramid image structure of reduced resolution versions of the image for solving the problems in the conventional methods is presented. This method is described the object detection and delineation by statistical approach. In the object detection method, IFSVR( Inverse-father-son variance ratio) method and FSVR(father-son variance ratio ) method are proposed for solving clustering validity problem occurred In the hierarchical pyramid image structure. An optimal object pixel Is detected at some level by this method. In the object delineation method, the iterative algorithm by top-down traversing method is proposed for moving the optimal object pixel to levels of higher resolution. Using the computer simulation, the results by the proposed statistical methods and object traversing method are investigated for the binary Image and the real image. At the results of computer simulation, the proposed methods of image segmentation based on the hierarchical pyramid Image structure seem to have useful properties and deserve consideration as a possible alternative to existing methods of image segmentation. The computation for the proposed method is required 0(log n) for n${\times}$n input image.

  • PDF

Merchandise Management Using Web Mining in Business To Customer Electronic Commerce (기업과 소비자간 전자상거래에서의 웹 마이닝을 이용한 상품관리)

  • 임광혁;홍한국;박상찬
    • Journal of Intelligence and Information Systems
    • /
    • v.7 no.1
    • /
    • pp.97-121
    • /
    • 2001
  • Until now, we have believed that one of advantages of cyber market is that it can virtually display and sell goods because it does not necessary maintain expensive physical shops and inventories. But, in a highly competitive environment, business model that does away with goods in stock must be modified. As we know in the case of AMAZON, leading companies already consider merchandise management as a critical success factor in their business model. That is, a solution to compete against one's competitors in a highly competitive environment is merchandise management as in the traditional retail market. Cyber market has not only past sales data but also web log data before sales data that contains information of path that customer search and purchase on cyber market as compared with traditional retail market. So if we can correctly analyze the characteristics of before sales patterns using web log data, we can better prepare for the potential customers and effectively manage inventories and merchandises. We introduce a systematic analysis method to extract useful data for merchandise management - demand forecasting, evaluating & selecting - using web mining that is the application of data mining techniques to the World Wide Web. We use various techniques of web mining such as clustering, mining association rules, mining sequential patterns.

  • PDF

User Perspective Website Clustering for Site Portfolio Construction (사이트 포트폴리오 구성을 위한 사용자 관점의 웹사이트 클러스터링)

  • Kim, Mingyu;Kim, Namgyu
    • Journal of Internet Computing and Services
    • /
    • v.16 no.3
    • /
    • pp.59-69
    • /
    • 2015
  • Many users visit websites every day to perform information retrieval, shopping, and community activities. On the other hand, there is intense competition among sites which attempt to profit from the Internet users. Thus, the owners or marketing officers of each site try to design a variety of marketing strategies including cooperation with other sites. Through such cooperation, a site can share customers' information, mileage points, and hyperlinks with other sites. To create effective cooperation, it is crucial to choose an appropriate partner site that may have many potential customers. Unfortunately, it is exceedingly difficult to identify such an appropriate partner among the vast number of sites. In this paper, therefore, we devise a new methodology for recommending appropriate partner sites to each site. For this purpose, we perform site clustering from the perspective of visitors' similarities, and then identify a group of sites that has a number of common customers. We then analyze the potential for the practical use of the proposed methodology through its application to approximately 140 million actual site browsing histories.

Development of a Railway Accident Scenario Analysis Technique using a Preliminary Hazard Analysis(PHA) and a Quality Function Deployment(QFD) (예비위험분석기술(PHA)과 품질기능전개(QFD) 기법을 이용한 철도사고 시나리오 분석기술 개발)

  • Park Chan-Woo;Kwak Sang-Log;Wang Jong-Bae;Hong Seong-Ho;Park Joo-Nam
    • Proceedings of the KSR Conference
    • /
    • 2005.05a
    • /
    • pp.151-156
    • /
    • 2005
  • The objective of this study is to devise an accident scenario analysis method adept at creating accident scenarios at the Preliminary Hazard Analysis(PHA) step of a hazard analysis for railway system. This approach was inspired by the Quality Function Deployment(QFD) method, which is conventionally used in quality management and was used at the systematic accident scenario analysis(SASA) for the design of safer products. In this study, the QFD provides a formal and systematic schema to devise accident scenarios while maintaining objective. The accident scenario analysis method first identifies the hazard factors that cause railway accidents and explains the situation characteristics surrounding the accident. This method includes a feasibility test, a clustering process and a pattering process for a clearer understanding of the accident situation. Since this method enables an accident scenario analysis method to be performed systematically as well as objectively, this method is useful in building better accident prevention strategies. Therefore, this study can serve to reduce railway accident and be an effective tool for a hazard analysis.

  • PDF