• Title/Summary/Keyword: Data Warehousing

Search Result 91, Processing Time 0.024 seconds

TATS: an Efficient Technique for Computing Temporal Aggregates for Data Warehousing

  • Shin, Young-Ok;Park, Sung-Kong;Baik, Doo-Kwon;Ryu, Keun-Ho
    • ETRI Journal
    • /
    • v.22 no.3
    • /
    • pp.41-51
    • /
    • 2000
  • An important use of data warehousing is to provide temporal views over the history of source data. It is significant that nearly all data warehouses are dependent on relational database technology, yet relational databases provide little or no real support for temporal data. Therefore, in is difficult to obtain accurate information for time-varying data. In this paper, we are going to design a temporal data warehouse to support time-varying data efficiently. For this purpose, we present a method to support temporal query by combining a temporal query process layer with the relational database which is used as a source database in an existing data warehouse. We introduce the Temporal Aggregate Tree Strategy (TATS), and suggest its algorithm for the way to aggregate the time-varying data that is changed by the time when the temporal view is created. In addition, The TATS and the materialized view creation method of the existing data warehouse have been evaluated. As a result, the TATS reduces the size of the fact table and it shows a good performance for the comparison factor in case of processing the query for time-varying data.

  • PDF

Extending the Multidimensional Data Model to Handle Complex Data

  • Mansmann, Svetlana;Scholl, Marc H.
    • Journal of Computing Science and Engineering
    • /
    • v.1 no.2
    • /
    • pp.125-160
    • /
    • 2007
  • Data Warehousing and OLAP (On-Line Analytical Processing) have turned into the key technology for comprehensive data analysis. Originally developed for the needs of decision support in business, data warehouses have proven to be an adequate solution for a variety of non-business applications and domains, such as government, research, and medicine. Analytical power of the OLAP technology comes from its underlying multidimensional data model, which allows users to see data from different perspectives. However, this model displays a number of deficiencies when applied to non-conventional scenarios and analysis tasks. This paper presents an attempt to systematically summarize various extensions of the original multidimensional data model that have been proposed by researchers and practitioners in the recent years. Presented concepts are arranged into a formal classification consisting of fact types, factual and fact-dimensional relationships, and dimension types, supplied with explanatory examples from real-world usage scenarios. Both the static elements of the model, such as types of fact and dimension hierarchy schemes, and dynamic features, such as support for advanced operators and derived elements. We also propose a semantically rich graphical notation called X-DFM that extends the popular Dimensional Fact Model by refining and modifying the set of constructs as to make it coherent with the formal model. An evaluation of our framework against a set of common modeling requirements summarizes the contribution.

A Conveyor Algorithm for Complete Consistency of Materialized View in a Self-Maintenance (실체 뷰의 자기관리에서 완전일관성을 위한 컨베이어 알고리듬)

  • Hong, In-Hoon;Kim, Yon-Soo
    • IE interfaces
    • /
    • v.16 no.2
    • /
    • pp.229-239
    • /
    • 2003
  • The On-Line Analytical Processing (OLAP) tools access data from the data warehouse for complex data analysis, such as multidimensional data analysis, and decision support activities. Current research has lead to new developments in all aspects of data warehousing, however, there are still a number of problems that need to be solved for making data warehousing effective. View maintenance, one of them, is to maintain view in response to updates in source data. Keeping the view consistent with updates to the base relations, however, can be expensive, since it may involve querying external sources where the base relations reside. In order to reduce maintenance costs, it is possible to maintain the views using information that is strictly local to the data warehouse. This process is usually referred to as "self-maintenance of views". A number of algorithm have been proposed for self maintenance of views where they keep some additional information in data warehouse in the form of auxiliary views. But those algorithms did not consider a consistency of materialized views using view self-maintenance. The purpose of this paper is to research consistency problem when self-maintenance of views is implemented. The proposed "conveyor algorithm" will resolved a complete consistency of materialized view using self-maintenance with considering network delay. The rationale for conveyor algorithm and performance characteristics are described in detail.

U.S. Port Investment Strategies and the Corresponding Economic Impacts Stemming from the Panama Canal Expansion

  • Park, ChangKeun
    • Asian Journal of Innovation and Policy
    • /
    • v.10 no.2
    • /
    • pp.195-211
    • /
    • 2021
  • This paper measures the economic impacts of the U.S. port investment strategies coping with the Panama Canal expansion. Using secondary import data, negative and positive estimates of the impacts were presented in this study. Reduced port activities into the West Coast Customs Districts negatively affect transportation and warehousing industries, among other effects. Still, they have simultaneous positive effects in other states from increased imports resulting from modal shifts and changes in the entry port located in the South and East coasts. This study applied the supply-driven National Interstate Economic Model that measures all interstate trade among the U.S. states to divert foreign imports from 15 Pacific Rim countries. For this purpose, the following assumption was adopted: larger ships using the canal will lead to a redirection of seaborne trade among U.S. (and other) ports and result in secondary effects, e.g., using different freight modes and regional growth spillovers. This study also accounted for the entry point change and significant port investments for foreign trade under alternative scenarios. The choice of ports for international trade depends on decisions about how to minimize multimodal delivery costs. The total direct reduction of transportation and warehousing activities associated with foreign imports in the West Coast ports was estimated at $3.3 billion, leading to total negative effects of $5.8 billion. Total positive impacts from the shift of transportation modes with the choice of an entry port and new warehousing activities for foreign imports in the selected 12 states varied. As expected, states that involved an entry port had the most prominent benefits, but Texas, New York, and New Jersey may be benefited through all the port enhancement projects in the U.S. Also, except for Transportation and Postal, and Warehousing industries, Construction is another dominant positive affected industry of the Canal expansion in the U.S.

Dangerous goods warehouse storage accident and safety management: evidence from Chinese data analysis (중국의 위험물 창고 보관사고 분석 및 안전관리방안에 관한 연구)

  • Miao Su;Yanfeng Liu;Du Siwen;Keun-sik Park
    • Korea Trade Review
    • /
    • v.46 no.4
    • /
    • pp.149-166
    • /
    • 2021
  • This paper aims to reduce the frequency of dangerous goods storage accidents in China. Advocating the managers of warehousing and logistics enterprises to pay attention to the operation process of dangerous goods warehousing business. Improving the safe storage and management capabilities of dangerous goods warehouses. This article first collects official data on dangerous goods storage accidents in China and conducts a general statistical analysis of the accidents. Based on the results of accident statistics and related literature research on dangerous goods storage management, establish a dangerous goods storage safety management factor system, use the analytic hierarchy process, establish a factor importance questionnaire and implement data collection. Through statistics, this paper finds that the storage accidents of dangerous goods in China in the past ten years mainly occurred in the inbound phase of dangerous goods and the storage phase of dangerous goods warehouses. Through the results of the analytic hierarchy process, it is found that the professionalism of the dangerous goods storage practitioners, the compliance of the practitioners with safety regulations, and the awareness of operational safety are the most important.

Creating Shared Value from Collaborative Logistics Systems: The Cases of ES3 and Flexe

  • Namchul Shin
    • Asia pacific journal of information systems
    • /
    • v.30 no.1
    • /
    • pp.214-228
    • /
    • 2020
  • Shared value enhances the competitiveness of a company while simultaneously reducing societal burdens. By allowing companies to share their resources, collaborative logistics systems provide companies with an opportunity to create shared value, namely, not only economic value by enhancing the utilization of resources, but also social value by reducing energy consumptions and greenhouse gas emissions associated with logistics and transportation. Emerging businesses, such as ES3 and Flexe, have recently demonstrated how they created shared value through collaborative logistics services, for example, ES3's collaborative warehousing and direct-to-store (D2S) program, and Flexe's on-demand warehousing platform. However, the development of collaborative logistics systems is currently at a nascent stage. There are quite a few socio-technical barriers to overcome for sharing resources (data as well as infrastructure). Drawing on the socio-technical approach, this research examines how companies create both economic and social value from collaborative logistics systems. We highlight socio-technical barriers, particularly one set of social barriers, that is, competition-oriented conservatism prevalent among companies. Using the case study methodology and interview data, we closely investigate ES3 and Flexe, which provide collaborative logistics services, and demonstrate how technical and social barriers are addressed to create shared value from collaborative logistics systems.

Research on Metadata Model of Data Warehouse

  • Zeng, Zhi-Yong;Yu, Jian-Kun
    • Proceedings of the Korea Society for Industrial Systems Conference
    • /
    • 2007.02a
    • /
    • pp.72-79
    • /
    • 2007
  • OMG's Common Warehouse Metamodel is now a single data warehouse metadata standard. Interchange of metadata between data warehousing tools and metadata repositories made easy and convenient by using CWM. In this paper, we present the origin, importance and architecture of CWM, and offer an application case.

  • PDF

Minimizing the MOLAP/ROLAP Divide: You Can Have Your Performance and Scale It Too

  • Eavis, Todd;Taleb, Ahmad
    • Journal of Computing Science and Engineering
    • /
    • v.7 no.1
    • /
    • pp.1-20
    • /
    • 2013
  • Over the past generation, data warehousing and online analytical processing (OLAP) applications have become the cornerstone of contemporary decision support environments. Typically, OLAP servers are implemented on top of either proprietary array-based storage engines (MOLAP) or as extensions to conventional relational DBMSs (ROLAP). While MOLAP systems do indeed provide impressive performance on common analytics queries, they tend to have limited scalability. Conversely, ROLAP's table oriented model scales quite nicely, but offers mediocre performance at best relative to the MOLAP systems. In this paper, we describe a storage and indexing framework that aims to provide both MOLAP like performance and ROLAP like scalability by essentially combining some of the best features from both. Based upon a combination of R-trees and bitmap indexes, the storage engine has been integrated with a robust OLAP query engine prototype that is able to fully exploit the efficiency of the proposed storage model. Specifically, it utilizes an OLAP algebra coupled with a domain specific query optimizer, to map user queries directly to the storage and indexing framework. Experimental results demonstrate that not only does the design improve upon more naive approaches, but that it does indeed offer the potential to optimize both query performance and scalability.

MAGRU: Multi-layer Attention with GRU for Logistics Warehousing Demand Prediction

  • Ran Tian;Bo Wang;Chu Wang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.3
    • /
    • pp.528-550
    • /
    • 2024
  • Warehousing demand prediction is an essential part of the supply chain, providing a fundamental basis for product manufacturing, replenishment, warehouse planning, etc. Existing forecasting methods cannot produce accurate forecasts since warehouse demand is affected by external factors such as holidays and seasons. Some aspects, such as consumer psychology and producer reputation, are challenging to quantify. The data can fluctuate widely or do not show obvious trend cycles. We introduce a new model for warehouse demand prediction called MAGRU, which stands for Multi-layer Attention with GRU. In the model, firstly, we perform the embedding operation on the input sequence to quantify the external influences; after that, we implement an encoder using GRU and the attention mechanism. The hidden state of GRU captures essential time series. In the decoder, we use attention again to select the key hidden states among all-time slices as the data to be fed into the GRU network. Experimental results show that this model has higher accuracy than RNN, LSTM, GRU, Prophet, XGboost, and DARNN. Using mean absolute error (MAE) and symmetric mean absolute percentage error(SMAPE) to evaluate the experimental results, MAGRU's MAE, RMSE, and SMAPE decreased by 7.65%, 10.03%, and 8.87% over GRU-LSTM, the current best model for solving this type of problem.