• Title/Summary/Keyword: multidimensional data processing

Search Result 92, Processing Time 0.025 seconds

A Study on an Automatic Classification Model for Facet-Based Multidimensional Analysis of Civil Complaints (패싯 기반 민원 다차원 분석을 위한 자동 분류 모델)

  • Na Rang Kim
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.29 no.1
    • /
    • pp.135-144
    • /
    • 2024
  • In this study, we propose an automatic classification model for quantitative multidimensional analysis based on facet theory to understand public opinions and demands on major issues through big data analysis. Civil complaints, as a form of public feedback, are generated by various individuals on multiple topics repeatedly and continuously in real-time, which can be challenging for officials to read and analyze efficiently. Specifically, our research introduces a new classification framework that utilizes facet theory and political analysis models to analyze the characteristics of citizen complaints and apply them to the policy-making process. Furthermore, to reduce administrative tasks related to complaint analysis and processing and to facilitate citizen policy participation, we employ deep learning to automatically extract and classify attributes based on the facet analysis framework. The results of this study are expected to provide important insights into understanding and analyzing the characteristics of big data related to citizen complaints, which can pave the way for future research in various fields beyond the public sector, such as education, industry, and healthcare, for quantifying unstructured data and utilizing multidimensional analysis. In practical terms, improving the processing system for large-scale electronic complaints and automation through deep learning can enhance the efficiency and responsiveness of complaint handling, and this approach can also be applied to text data processing in other fields.

An Assignment Method of Multidimensional Type Inheritance Indexes for XML Query Processing (XML 질의처리를 위한 다차원 타입상속 색인구조의 할당기법)

  • Lee, Jong-Hak
    • Journal of Korea Multimedia Society
    • /
    • v.12 no.1
    • /
    • pp.1-15
    • /
    • 2009
  • This paper presents an assignment method of the multidimensional type inheritance indexes (MD-TIXs) to support the processing of XML queries in XML databases. MD-TIX uses a multidimensional index structure for efficiently supporting nested predicates that involve both nested element and type inheritance hierarchies. In this paper, we have analyzed the strategy of the query processing by using the MD-TIXs, and presented an assignment method of the MD-TIXs in the framework of complex queries, containing conjunctions of nested predicates, each one involving an Xpath having target types or domain types substitution. We first consider MD-TIX operations caused by updating of XML data-bases, and the use of the MD-TIXs in the case of a query containing a single nested predicate. And then, we consider the assignments of the MD-TIXs in the framework of more general queries containing nested predicates over overlapping paths that have common subpaths.

  • PDF

A Physical Design Method of Storage Structures for MOLAP Systems of Data Warehouse (데이터 웨어하우스의 다차원 온라인 분석처리 시스템을 위한 저장구조의 물리적 설계기법)

  • Lee Jong-Hak
    • Journal of Korea Multimedia Society
    • /
    • v.8 no.3
    • /
    • pp.297-312
    • /
    • 2005
  • Aggregation is an operation that plays a key role in multidimensional OLAP (MOLAP) systems of data warehouse. Existing aggregation operations in MOLAP have been proposed for file structures such as multidimensional arrays. These tile structures do not work well with skewed distributions. This paper presents a physical design methodology for storage structures ni MOLAP that use the multidimensional tile organizations adapting to a skewed distribution. In uniform data distribution, we first show that the performance of multidimensional analytical processing is highly affected by the similarity of the shapes between query regions and page regions in the domain space of the multidimensional file organizations. And than, in skewed distributions, we reflect the effect of data distributions on the design by using the shapes of the normalized query regions that are weighted with data density of those query regions. Finally, we demonstrate that the physical design methodology theoretically derived is indeed correct in real environments. In the two-dimensional file organizations, the results of experiments indicate that the performance of the proposed method is enhanced by more than seven times over the conventional method. We expect that the performance will be more enhanced when the dimensionality is more than two. The result confirms that the proposed physical design methodology is useful in a practical way.

  • PDF

Efficient Processing of Multidimensional Sensor stream Data in Digital Marine Vessel (디지털 선박 내 다차원 센서 스트림 데이터의 효율적인 처리)

  • Song, Byoung-Ho;Park, Kyung-Woo;Lee, Jin-Seok;Lee, Keong-Hyo;Jung, Min-A;Lee, Sung-Ro
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.35 no.5B
    • /
    • pp.794-800
    • /
    • 2010
  • It is necessary to accurate and efficient management for measured digital data from various sensors in digital marine vessel. It is not efficient that sensor network process input stream data of mass storage stored in database the same time. In this paper, We propose to improve the processing performance of multidimensional stream data continuous incoming from multiple sensor. We propose that we arrange some sensors (temperature, humidity, lighting, voice) and process query based on sliding window for efficient input stream and found multiple query plan to Mjoin method and we reduce stored data using SVM algorithm. We automatically delete that it isn't necessary to the data from the database and we used to ship diagnosis system for available data. As a result, we obtained to efficient result about 18.3% reduction rate of database using 35,912 data sets.

An Efficient Query Transformation for Multidimensional Data Views on Relational Databases (관계형 데이타베이스에서 다차원 데이타의 뷰를 위한 효율적인 질의 변환)

  • Shin, Sung-Hyun;Kim, Jin-Ho;Moon, Yang-Sae
    • Journal of KIISE:Databases
    • /
    • v.34 no.1
    • /
    • pp.18-34
    • /
    • 2007
  • In order to provide various business analysis methods, OLAP(On-Line Analytical Processing) systems represent their data with multidimensional structures. These multidimensional data are often delivered to users in the horizontal format of tables whose columns are corresponding to values of dimension attributes. Since the horizontal tables nay have a large number of columns, they cannot be stored directly in relational database systems. Furthermore, the tables are likely to have many null values (i.e., sparse tables). In order to manage the horizontal tables efficiently, we can store them as the vertical format of tables which has dimension attribute names as their columns thus transforms the columns of horizontal tables into rows. In this way, every queries for horizontal tables have to be transformed into those for vertical tables. This paper proposed a technique for transforming horizontal table queries into vertical table ones by utilizing not only traditional relational algebraic operators but also the PIVOT operator which recent DBMS versions are providing. For achieving this goal, we designed a relational algebraic expression equivalent to the PIVOT operator and we formally proved their equivalence. Then, we developed a transformation technique for horizontal table queries using the PIVOT operator. We also performed experiments to analyze the performance of the proposed method. From the experimental results, we revealed that the proposed method has better performance than existing methods.

Sort-Based Distributed Parallel Data Cube Computation Algorithm using MapReduce (맵리듀스를 이용한 정렬 기반의 데이터 큐브 분산 병렬 계산 알고리즘)

  • Lee, Suan;Kim, Jinho
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.49 no.9
    • /
    • pp.196-204
    • /
    • 2012
  • Recently, many applications perform OLAP(On-Line Analytical Processing) over a very large volume of data. Multidimensional data cube is regarded as a core tool in OLAP analysis. This paper focuses on the method how to efficiently compute data cubes in parallel by using a popular parallel processing tool, MapReduce. We investigate efficient ways to implement PipeSort algorithm, a well-known data cube computation method, on the MapReduce framework. The PipeSort executes several (descendant) cuboids at the same time as a pipeline by scanning one (ancestor) cuboid once, which have the same sorting order. This paper proposed four ways implementing the pipeline of the PipeSort on the MapReduce framework which runs across 20 servers. Our experiments show that PipeMap-NoReduce algorithm outperforms the rest algorithms for high-dimensional data. On the contrary, Post-Pipe stands out above the others for low-dimensional data.

Noise Suppression of NMR Signal by Piecewise Polynomial Truncated Singular Value Decomposition

  • Kim, Daesung;Youngdo Won;Hoshik Won
    • Journal of the Korean Magnetic Resonance Society
    • /
    • v.4 no.2
    • /
    • pp.116-124
    • /
    • 2000
  • Singular value decomposition (SVD) has been used during past few decades in the advanced NMR data processing and in many applicable areas. A new modified SVD, piecewise polynomial truncated SVD (PPTSVD) was developed far the large solvent peak suppression and noise elimination in U signal processing. PPTSVD consists of two algorithms of truncated SVD (TSVD) and L$_1$ problems. In TSVD, some unwanted large solvent peaks and noises are suppressed with a certain son threshold value while signal and noise in raw data are resolved and eliminated out in L$_1$ problem routine. The advantage of the current PPTSVD method compared to many SVD methods is to give the better S/N ratio in spectrum, and less time consuming job that can be applicable to multidimensional NMR data processing.

  • PDF

The Framework of Stream Data Processing System for Realtime Health Care Service (실시간 헬스케어 서비스를 위한 스트림 데이터 시스템 프레임워크의 설계)

  • Wu, Zejun;Lee, Yeon;Bae, Hae-Young
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2011.06a
    • /
    • pp.21-22
    • /
    • 2011
  • The growth of using smartphone and tablet pc has enabled variety kinds of realtime applications. In these applications, the data which we called data stream is multidimensional, continuous, rapid, and time-varying. However the traditional Database Management System (DBMS) suffers from processing the real time and complex application, in this paper we proposed the framework for CCR Data Stream Server's design and implementation that compiled with Data Stream Database Management System (DSMS) and DBMS in EMR system. The system enables users not only to query stored CCR information from DBMS, but also to execute continues query for the real-time CCR Data Stream.

  • PDF

VDCluster : A Video Segmentation and Clustering Algorithm for Large Video Sequences (VDCluster : 대용량 비디오 시퀀스를 위한 비디오 세그멘테이션 및 클러스터링 알고리즘)

  • Lee, Seok-Ryong;Lee, Ju-Hong;Kim, Deok-Hwan;Jeong, Jin-Wan
    • Journal of KIISE:Databases
    • /
    • v.29 no.3
    • /
    • pp.168-179
    • /
    • 2002
  • In this paper, we investigate video representation techniques that are the foundational work for the subsequent video processing such as video storage and retrieval. A video data set if a collection of video clips, each of which is a sequence of video frames and is represented by a multidimensional data sequence (MDS). An MDS is partitioned into video segments considering temporal relationship among frames, and then similar segments of the clip are grouped into video clusters. Thus, the video clip is represented by a small number of video clusters. The video segmentation and clustering algorithm, VDCluster, proposed in this paper guarantee clustering quality to south an extent that satisfies predefined conditions. The experiments show that our algorithm performs very effectively with respect to various video data sets.

An Efficient ROLAP Cube Generation Scheme (효율적인 ROLAP 큐브 생성 방법)

  • Kim, Myung;Song, Ji-Sook
    • Journal of KIISE:Databases
    • /
    • v.29 no.2
    • /
    • pp.99-109
    • /
    • 2002
  • ROLAP(Relational Online Analytical Processing) is a process and methodology for a multidimensional data analysis that is essential to extract desired data and to derive value-added information from an enterprise data warehouse. In order to speed up query processing, most ROLAP systems pre-compute summary tables. This process is called 'cube generation' and it mostly involves intensive table sorting stages. (1) showed that it is much faster to generate ROLAP summary tables indirectly using a MOLAP(multidimensional OLAP) cube generation algorithm. In this paper, we present such an indirect ROLAP cube generation algorithm that is fast and scalable. High memory utilization is achieved by slicing the input fact table along one or more dimensions before generating summary tables. High speed is achieved by producing summary tables from their smallest parents. We showed the efficiency of our algorithm through experiments.