• Title/Summary/Keyword: top-k query processing

Search Result 38, Processing Time 0.027 seconds

Minimizing the MOLAP/ROLAP Divide: You Can Have Your Performance and Scale It Too

  • Eavis, Todd;Taleb, Ahmad
    • Journal of Computing Science and Engineering
    • /
    • v.7 no.1
    • /
    • pp.1-20
    • /
    • 2013
  • Over the past generation, data warehousing and online analytical processing (OLAP) applications have become the cornerstone of contemporary decision support environments. Typically, OLAP servers are implemented on top of either proprietary array-based storage engines (MOLAP) or as extensions to conventional relational DBMSs (ROLAP). While MOLAP systems do indeed provide impressive performance on common analytics queries, they tend to have limited scalability. Conversely, ROLAP's table oriented model scales quite nicely, but offers mediocre performance at best relative to the MOLAP systems. In this paper, we describe a storage and indexing framework that aims to provide both MOLAP like performance and ROLAP like scalability by essentially combining some of the best features from both. Based upon a combination of R-trees and bitmap indexes, the storage engine has been integrated with a robust OLAP query engine prototype that is able to fully exploit the efficiency of the proposed storage model. Specifically, it utilizes an OLAP algebra coupled with a domain specific query optimizer, to map user queries directly to the storage and indexing framework. Experimental results demonstrate that not only does the design improve upon more naive approaches, but that it does indeed offer the potential to optimize both query performance and scalability.

Multi-dimensional Traveling salesman problem using Top-n Skyline query (Top-n 스카이라인 질의를 이용한 다차원 외판원 순회문제)

  • Jin, ChangGyun;Yang, Sevin;Kang, Eunjin;Kim, JiYun;Kim, Jongwan;Oh, Dukshin
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2019.05a
    • /
    • pp.371-374
    • /
    • 2019
  • PDA나 휴대폰 단말로 여러 속성의 데이터를 이용하여 사용자에게 필요한 정보를 제공하는 위치기반 서비스는 물류/운송 정보 서비스, 버스/지하철 노선 안내 서비스 등에 사용된다. 여기에서 제공하는 데이터들을 최적 경로를 구하는 외판원 순회문제 (Traveling Salesman Problem)에 사용한다면 더 정확한 경로 서비스 제공이 가능하다. 하지만 데이터의 수가 많아질수록 비교 횟수가 기하급수적으로 늘어나는 외판원 순회 알고리즘의 특성상 일반 단말기에서 활용하기에는 배터리의 제약이 따른다. 본 논문에서는 이와 같은 단점을 해결하기 위해서 최적 경로의 후보군을 줄일 수 있는 스카이라인 질의를 이용하여 n차원 속성에 대한 최적 경로 알고리즘을 제안한다. 실험에서 정확도와 오차율을 통해 제안한 방식의 유용성을 보였으며 기존방식과 연산시간 차이를 비교하여 다차원방식의 효율성을 나타내었다.

OLAP4R: A Top-K Recommendation System for OLAP Sessions

  • Yuan, Youwei;Chen, Weixin;Han, Guangjie;Jia, Gangyong
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.6
    • /
    • pp.2963-2978
    • /
    • 2017
  • The Top-K query is currently played a key role in a wide range of road network, decision making and quantitative financial research. In this paper, a Top-K recommendation algorithm is proposed to solve the cold-start problem and a tag generating method is put forward to enhance the semantic understanding of the OLAP session. In addition, a recommendation system for OLAP sessions called "OLAP4R" is designed using collaborative filtering technique aiming at guiding the user to find the ultimate goals by interactive queries. OLAP4R utilizes a mixed system architecture consisting of multiple functional modules, which have a high extension capability to support additional functions. This system structure allows the user to configure multi-dimensional hierarchies and desirable measures to analyze the specific requirement and gives recommendations with forthright responses. Experimental results show that our method has raised 20% recall of the recommendations comparing the traditional collaborative filtering and a visualization tag of the recommended sessions will be provided with modified changes for the user to understand.

Medical Image Classification and Retrieval Using BoF Feature Histogram with Random Forest Classifier (Random Forest 분류기와 Bag-of-Feature 특징 히스토그램을 이용한 의료영상 자동 분류 및 검색)

  • Son, Jung Eun;Ko, Byoung Chul;Nam, Jae Yeal
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.4
    • /
    • pp.273-280
    • /
    • 2013
  • This paper presents novel OCS-LBP (Oriented Center Symmetric Local Binary Patterns) based on orientation of pixel gradient and image retrieval system based on BoF (Bag-of-Feature) and random forest classifier. Feature vectors extracted from training data are clustered into code book and each feature is transformed new BoF feature using code book. BoF features are applied to random forest for training and random forest having N classes is constructed by combining several decision trees. For testing, the same OCS-LBP feature is extracted from a query image and BoF is applied to trained random forest classifier. In contrast to conventional retrieval system, query image selects similar K-nearest neighbor (K-NN) classes after random forest is performed. Then, Top K similar images are retrieved from database images that are only labeled K-NN classes. Compared with other retrieval algorithms, the proposed method shows both fast processing time and improved retrieval performance.

Personalized Itinerary Recommendation System based on Stay Time (체류시간을 고려한 여행 일정 추천 시스템)

  • Park, Sehwa;Park, Seog
    • KIISE Transactions on Computing Practices
    • /
    • v.22 no.1
    • /
    • pp.38-43
    • /
    • 2016
  • Recent developments regarding transportation technology have positioned travel as a major leisure activity; however, trip-itinerary planning remains a challenging task for tourists due to the need to select Points of Interest (POI) for visits to unfamiliar cities. Meanwhile, due to the GPS functions on mobile devices such as smartphones and tablet PCs, it is now possible to collect a user's position in real time. Based on these circumstances, our research on an automatic itinerary-planning system to simplify the trip-planning process was conducted briskly. The existing studies that include research on itinerary schedules focus on an identification of the shortest path in consideration of cost and time constraints, or a recommendation of the most-popular travel route in the destination area; therefore, we propose a personalized itinerary-recommendation system for which the stay-time preference of the individual user is considered as part of the personalized service.

인터넷 질의 처리를 위한 웨이블릿 변환에 기반한 통합 요약정보의 관리

  • Joe, Moon-Jeung;Whang, Kyu-Young;Kim, Sang-Wook;Shim, Kyu-Seok
    • Journal of KIISE:Databases
    • /
    • v.28 no.4
    • /
    • pp.702-714
    • /
    • 2001
  • As Internet technology evolves, there is growing need of Internet queries involving multiple information sources. Efficient processing of such queries necessitates the integrated summary data that compactly represents the data distribution of the entire database scattered over many information sources. This paper presents an efficient method of managing the integrated summary data based on the wavelet transform and addresses Internet query processing using the integrated summary data. The simplest method for creating the integrated summary data would be to summarize the integrated data sidtribution obtained by merging the data distributions in multiple information sources. However, this method suffers from the high cost of transmitting storing and merging a large amount of data distribution. To overcome the drawbacks, we propose a new wavelet transform based method that creates the integrated summary data by merging multiple summary data and effective method for optimizing Internet queries using it A wavelet transformed summary data is converted to satisfy conditions for merging. Moreover i the merging process is very simpe owing to the properties of the wavelet transform. we formally derive the upper bound of the error of the wavelet transformed intergrated summary data. Compared with the histogram-based integrated summary data the wavelet transformedintegrated summary data provesto be 1.6~5.5 time more accurate when used for selectivity estimation in experiments. In processing Internet top-N queries involving 56 information sources using the integrated summary data reduces the processing cost to 1/44 of the cost of not using it.

  • PDF

An XML-QL to SQL Translator for Processing XML Data (XML 데이타 처리를 위한 XML-QL to SQL 번역기)

  • Jang, Gyeong-Ja;Lee, Gi-Ho
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.8 no.1
    • /
    • pp.1-8
    • /
    • 2002
  • XML has been proposed as an international standard for organizing and exchanging a great diversity of the Web data. It is important to retrieve components of stored XML documents that are needed by a wide variety of applications. In this paper, we suggest a method to store XML documents and to retrieve an XML data. In other words, we suggest the method of retrieving XML data is using XML -QL. So we need to mapping XML-QL to SQL translator on top of an RDBMS. The contributions of this paper include, besides the detailed design and implementation of the translator, demonstration of feasibility of such a translator, and a comprehensive classification of XML queries and their mappings to SQL relational queries.

Methods for Integration of Documents using Hierarchical Structure based on the Formal Concept Analysis (FCA 기반 계층적 구조를 이용한 문서 통합 기법)

  • Kim, Tae-Hwan;Jeon, Ho-Cheol;Choi, Joong-Min
    • Journal of Intelligence and Information Systems
    • /
    • v.17 no.3
    • /
    • pp.63-77
    • /
    • 2011
  • The World Wide Web is a very large distributed digital information space. From its origins in 1991, the web has grown to encompass diverse information resources as personal home pasges, online digital libraries and virtual museums. Some estimates suggest that the web currently includes over 500 billion pages in the deep web. The ability to search and retrieve information from the web efficiently and effectively is an enabling technology for realizing its full potential. With powerful workstations and parallel processing technology, efficiency is not a bottleneck. In fact, some existing search tools sift through gigabyte.syze precompiled web indexes in a fraction of a second. But retrieval effectiveness is a different matter. Current search tools retrieve too many documents, of which only a small fraction are relevant to the user query. Furthermore, the most relevant documents do not nessarily appear at the top of the query output order. Also, current search tools can not retrieve the documents related with retrieved document from gigantic amount of documents. The most important problem for lots of current searching systems is to increase the quality of search. It means to provide related documents or decrease the number of unrelated documents as low as possible in the results of search. For this problem, CiteSeer proposed the ACI (Autonomous Citation Indexing) of the articles on the World Wide Web. A "citation index" indexes the links between articles that researchers make when they cite other articles. Citation indexes are very useful for a number of purposes, including literature search and analysis of the academic literature. For details of this work, references contained in academic articles are used to give credit to previous work in the literature and provide a link between the "citing" and "cited" articles. A citation index indexes the citations that an article makes, linking the articleswith the cited works. Citation indexes were originally designed mainly for information retrieval. The citation links allow navigating the literature in unique ways. Papers can be located independent of language, and words in thetitle, keywords or document. A citation index allows navigation backward in time (the list of cited articles) and forwardin time (which subsequent articles cite the current article?) But CiteSeer can not indexes the links between articles that researchers doesn't make. Because it indexes the links between articles that only researchers make when they cite other articles. Also, CiteSeer is not easy to scalability. Because CiteSeer can not indexes the links between articles that researchers doesn't make. All these problems make us orient for designing more effective search system. This paper shows a method that extracts subject and predicate per each sentence in documents. A document will be changed into the tabular form that extracted predicate checked value of possible subject and object. We make a hierarchical graph of a document using the table and then integrate graphs of documents. The graph of entire documents calculates the area of document as compared with integrated documents. We mark relation among the documents as compared with the area of documents. Also it proposes a method for structural integration of documents that retrieves documents from the graph. It makes that the user can find information easier. We compared the performance of the proposed approaches with lucene search engine using the formulas for ranking. As a result, the F.measure is about 60% and it is better as about 15%.