• 제목/요약/키워드: Data Sets

검색결과 3,753건 처리시간 0.027초

연속발생 데이터를 위한 실시간 데이터 마이닝 기법 (A Real-Time Data Mining for Stream Data Sets)

  • 김진화;민진영
    • 한국경영과학회지
    • /
    • 제29권4호
    • /
    • pp.41-60
    • /
    • 2004
  • A stream data is a data set that is accumulated to the data storage from a data source over time continuously. The size of this data set, in many cases. becomes increasingly large over time. To mine information from this massive data. it takes much resource such as storage, memory and time. These unique characteristics of the stream data make it difficult and expensive to use this large size data accumulated over time. Otherwise. if we use only recent or part of a whole data to mine information or pattern. there can be loss of information. which may be useful. To avoid this problem. we suggest a method that efficiently accumulates information. in the form of rule sets. over time. It takes much smaller storage compared to traditional mining methods. These accumulated rule sets are used as prediction models in the future. Based on theories of ensemble approaches. combination of many prediction models. in the form of systematically merged rule sets in this study. is better than one prediction model in performance. This study uses a customer data set that predicts buying power of customers based on their information. This study tests the performance of the suggested method with the data set alone with general prediction methods and compares performances of them.

Introduction of Japanese Ocean Flux data sets with Use of Remote sensing Observations (J-OFURO)

  • Kubota, Masahisa
    • 대한원격탐사학회:학술대회논문집
    • /
    • 대한원격탐사학회 1999년도 Proceedings of International Symposium on Remote Sensing
    • /
    • pp.231-236
    • /
    • 1999
  • Accurate ocean surface fluxes with high resolution are critical for understanding a mechanism of global climate. However, it is difficult to derive those fluxes by using ocean observation data because the number of ocean observation data is extremely small and the distribution is inhomogeneous. On the other hand. satellite data are characterized by the high density, the high resolution and the homogeneity. Therefore, it can be considered that we obtain accurate ocean surface by using satellite data. Recently we constructed ocean surface data sets mainly using satellite data. The data set is named by Japanese Ocean Flux data sets with Use of Remote sensing Observations (J-OFURO). Here, we introduce J-OFURO. The data set includes shortwave radiation, longwave radiation, latent heat flux, sensible heat flux, and momentum flux etc. Moreover, sea surface dynamic topography data are included in the data set. Radiation data sets covers western Pacific and eastern Indian Ocean because we use a Japanese geostationally satellite (GMS) to estimate radiation fluxes. On the other hand, turbulent heat fluxes are globally estimated. The constructed data sets are used and shows the effectiveness for many scientific studies.

  • PDF

Wavenumber correlation analysis of satellite magnetometer observations

  • Kim, Jeong-Woo;Kim, Won-Kyun;Kim, Hye-Yun
    • 대한자원환경지질학회:학술대회논문집
    • /
    • 대한자원환경지질학회 2000년도 춘계공동학술발표회
    • /
    • pp.311-313
    • /
    • 2000
  • Identifying anomaly correlations between data sets is the basis for rationalizing geopotenial interpretation and theory. A procedure between the two or more geopotential data sets. Anomaly features that show direct, inverse, or no correlationsbetween the data may be separated by applying filters in the frequency domains of the data sets. The correlation filter passes or rejects wavenumbers between co-registered data sets based on the correlation coefficient between common wavenumbers as given by the cosine of their phase difference. This study includes as example of Magsat magnetic anomaly profile that illustrates the usefulness of the procedure for extracting correlative features between the sets.

  • PDF

Efficient Continuous Skyline Query Processing Scheme over Large Dynamic Data Sets

  • Li, He;Yoo, Jaesoo
    • ETRI Journal
    • /
    • 제38권6호
    • /
    • pp.1197-1206
    • /
    • 2016
  • Performing continuous skyline queries of dynamic data sets is now more challenging as the sizes of data sets increase and as they become more volatile due to the increase in dynamic updates. Although previous work proposed support for such queries, their efficiency was restricted to small data sets or uniformly distributed data sets. In a production database with many concurrent queries, the execution of continuous skyline queries impacts query performance due to update requirements to acquire exclusive locks, possibly blocking other query threads. Thus, the computational costs increase. In order to minimize computational requirements, we propose a method based on a multi-layer grid structure. First, relational data object, elements of an initial data set, are processed to obtain the corresponding multi-layer grid structure and the skyline influence regions over the data. Then, the dynamic data are processed only when they are identified within the skyline influence regions. Therefore, a large amount of computation can be pruned by adopting the proposed multi-layer grid structure. Using a variety of datasets, the performance evaluation confirms the efficiency of the proposed method.

상품간 연관 규칙의 효율적 탐색 방법에 관한 연구 : 인터넷 쇼핑몰을 중심으로 (A Fast Algorithm for Mining Association Rules in Web Log Data)

  • 오은정;오상봉
    • 한국산업정보학회:학술대회논문집
    • /
    • 한국산업정보학회 2003년도 추계공동학술대회
    • /
    • pp.621-626
    • /
    • 2003
  • Mining association rules in web log files can be divided into two steps: 1) discovering frequent item sets in web data; 2) extracting association rules from the frequent item sets found in the previous step. This paper suggests an algorithm for finding frequent item sets efficiently The essence of the proposed algorithm is to transform transaction data files into matrix format. Our experimental results show that the suggested algorithm outperforms the Apriori algorithm, which is widely used to discover frequent item sets, in terms of scan frequency and execution time.

  • PDF

형식 개념 분석을 통한 공공데이터의 메타데이터 분석 (Metadata Analysis of Open Government Data by Formal Concept Analysis)

  • 김학래
    • 한국콘텐츠학회논문지
    • /
    • 제18권1호
    • /
    • pp.305-313
    • /
    • 2018
  • 공공데이터는 공공기관이 만들어내는 자료나 정보를 국민에게 공개한 것이다. 정부는 공공데이터포털과 개별기관의 웹사이트를 통해 공공데이터를 개방하고 있다. 그러나 데이터 사용자 관점에서 원하는 공공데이터를 탐색하고 활용하는데 제약이 있는 것이 현실이다. 특히, 데이터 목록의 특성을 파악하고 서로 다른 데이터를 연계하는 과정에 많은 노력과 시간이 필요하다. 본 연구는 공공데이터로 개방된 데이터 목록이 갖고 있는 항목명의 공통 관계를 분석하여 데이터 목록사이의 연결 가능성을 제안한다. 공공데이터포털에서 제공하는 데이터 목록을 수집하고, 데이터 목록에 포함된 데이터 항목명을 추출한다. 추출된 항목명은 형식 개념 분석을 통해 형식 문맥 (formal context)과 형식 개념 (formal concept)으로 구성된다. 형식 개념은 데이터 목록과 항목명을 각각 외연과 내연으로 갖고 있고, 내연의 공통항목을 분석해 데이터 연결 가능성을 판별한다. 형식 개념 분석을 통해 도출한 결과는 데이터 목록의 의미적 연결에 효과적으로 활용될 수 있고, 공공데이터 개방을 위한 데이터 표준 및 품질개선에 적용할 수 있다.

다중 지구과학자료를 이용한 GIS 기반 공간통합과 통계량 분석 : 광물 부존 예상도 작성을 위한 사례 연구 (GIS-based Spatial Integration and Statistical Analysis using Multiple Geoscience Data Sets : A Case Study for Mineral Potential Mapping)

  • 이기원;박노욱;권병두;지광훈
    • 대한원격탐사학회지
    • /
    • 제15권2호
    • /
    • pp.91-105
    • /
    • 1999
  • 최근 다중 지질정보의 통합적 해석은 GIS의 중요한 응용 분야중 하나로 인식되고 있다. 공간통합을 위하여 지구통계학적 방법들이 개발되어 있지만, 통합결과와 입력 주제도들 사이의 관계에 대한 통계적, 정량적 분석방법론의 개발은 아직까지 체계적으로 정립되어 있지 못한 상황이다. 본 연구에서는 지질도, 지화학자료, 항공지구물리자료, 지형자료 및 원격탐사 영상등 다양한 지질정보등이 보고된 옥동지역을 대상으로 하여 광물 부존 예상도 작성 사례연구를 수행하여 기존에 이용되고 있는 여러 공간 통합 방법중 확실인자 (Certainty Factor: CF) 추정방법과 다변량 통계 분석방법중 하나인 주성분분석을 시험적인 통합방법으로 우선적으로 적용한 뒤, 입력 자료와 통합결과에 대한 정량적인 통계량 정보를 추출하고자 하였다. 입력 주제도와 통합 결과사이의 관계 규명에는 통계 분할표를 이용한 통계처리를 편의 분석에는 잭나이프 방법을 적용하였다. 통합정보에 대한 통계량 분석을 통하여, 통합 결과와 입력자료 사이의 정량적 관계를 추출할 수 있었으며, 부가적으로 입력자료의 상태수준에 대한 판단정보를 얻을 수 있었다. 이러한 결과는 GIS 관점에서 통합결과 해석에 중요한 결정보조자료로 활용될 수 있으며, 복잡한 다중정보를 다루는데 공간 통합문제에서도 입력정보 검증을 위한 일반적일 처리과정으로도 발전할 수 있을 것으로 생각된다.

A Measure of Agreement for Multivariate Interval Observations by Different Sets of Raters

  • Um, Yong-Hwan
    • Journal of the Korean Data and Information Science Society
    • /
    • 제15권4호
    • /
    • pp.957-963
    • /
    • 2004
  • A new agreement measure for multivariate interval data by different sets of raters is proposed. The proposed approach builds on Um's multivariate extension of Cohen's kappa. The proposed measure is compared with corresponding earlier measures based on Berry and Mielke's approach and Janson and Olsson approach, respectively. Application of the proposed measure is exemplified using hypothetical data set.

  • PDF

Parameter Estimation and Prediction for NHPP Software Reliability Model and Time Series Regression in Software Failure Data

  • Song, Kwang-Yoon;Chang, In-Hong
    • 통합자연과학논문집
    • /
    • 제7권1호
    • /
    • pp.67-73
    • /
    • 2014
  • We consider the mean value function for NHPP software reliability model and time series regression model in software failure data. We estimate parameters for the proposed models from two data sets. The values of SSE and MSE is presented from two data sets. We compare the predicted number of faults with the actual two data sets using the mean value function and regression curve.

A Knowledge Discovery Framework for Spatiotemporal Data Mining

  • Lee, Jun-Wook;Lee, Yong-Joon
    • Journal of Information Processing Systems
    • /
    • 제2권2호
    • /
    • pp.124-129
    • /
    • 2006
  • With the explosive increase in the generation and utilization of spatiotemporal data sets, many research efforts have been focused on the efficient handling of the large volume of spatiotemporal sets. With the remarkable growth of ubiquitous computing technology, mining from the huge volume of spatiotemporal data sets is regarded as a core technology which can provide real world applications with intelligence. In this paper, we propose a 3-tier knowledge discovery framework for spatiotemporal data mining. This framework provides a foundation model not only to define the problem of spatiotemporal knowledge discovery but also to represent new knowledge and its relationships. Using the proposed knowledge discovery framework, we can easily formalize spatiotemporal data mining problems. The representation model is very useful in modeling the basic elements and the relationships between the objects in spatiotemporal data sets, information and knowledge.