• Title/Summary/Keyword: Grouping Stream

Search Result 26, Processing Time 0.029 seconds

Dynamic Data Cubes Over Data Streams (데이타 스트림에서 동적 데이타 큐브)

  • Seo, Dae-Hong;Yang, Woo-Sock;Lee, Won-Suk
    • Journal of KIISE:Databases
    • /
    • v.35 no.4
    • /
    • pp.319-332
    • /
    • 2008
  • Data cube, which is multi-dimensional data model, have been successfully applied in many cases of multi-dimensional data analysis, and is still being researched to be applied in data stream analysis. Data stream is being generated in real-time, incessant, immense, and volatile manner. The distribution characteristics of data arc changing rapidly due to those characteristics, so the primary rule of handling data stream is to check once and dispose it. For those characteristics, users are more interested in high support attribute values observed rather than the entire attribute values over data streams. This paper propose dynamic data cube for applying data cube to data stream environment. Dynamic data cube specify user's interested area by the support ratio of attribute value, and dynamically manage the attribute values by grouping each other. By doing this it reduce the memory usage and process time. And it can efficiently shows or emphasize user's interested area by increasing the granularity for attributes that have higher support. We perform experiments to verify how efficiently dynamic data cube works in limited memory usage.

A method of event data stream processing for ALE Middleware (ALE 미들웨어를 위한 이벤트 데이터 처리 방법)

  • Noh, Young-Sik;Lee, Dong-Cheol;Byun, Yung-Cheol
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.12 no.9
    • /
    • pp.1554-1563
    • /
    • 2008
  • As the interests on RFID technologies increase, a lot of research activities on RFID middleware systems to handle the data acquired by RFID readers are going on actively. Meanwhile, even though various kinds of RFID middleware methodologies and related techniques have been proposed, the common data type which is dealt with in those systems is an EPC code, mainly. Also, there are few researches of the implementation of collecting the stream data queued from RFID readers endlessly and without blocking, classifying the data into some groups according to usage, and sending the resulting data to specific applications. In this paper, we propose the method of data handling in RFID middleware to efficiently process an EPC event stream data using detail filtering, checking of data modification, creation of data set to transfer, data grouping, and various kinds of RFID data format transform. Our method is based on a de facto international standard interface defined in the ALE middleware specification by EPCglobal, and application and service users can directly set various kinds of conditions to handle the stream data.

A Multiple Instance Learning Problem Approach Model to Anomaly Network Intrusion Detection

  • Weon, Ill-Young;Song, Doo-Heon;Ko, Sung-Bum;Lee, Chang-Hoon
    • Journal of Information Processing Systems
    • /
    • v.1 no.1 s.1
    • /
    • pp.14-21
    • /
    • 2005
  • Even though mainly statistical methods have been used in anomaly network intrusion detection, to detect various attack types, machine learning based anomaly detection was introduced. Machine learning based anomaly detection started from research applying traditional learning algorithms of artificial intelligence to intrusion detection. However, detection rates of these methods are not satisfactory. Especially, high false positive and repeated alarms about the same attack are problems. The main reason for this is that one packet is used as a basic learning unit. Most attacks consist of more than one packet. In addition, an attack does not lead to a consecutive packet stream. Therefore, with grouping of related packets, a new approach of group-based learning and detection is needed. This type of approach is similar to that of multiple-instance problems in the artificial intelligence community, which cannot clearly classify one instance, but classification of a group is possible. We suggest group generation algorithm grouping related packets, and a learning algorithm based on a unit of such group. To verify the usefulness of the suggested algorithm, 1998 DARPA data was used and the results show that our approach is quite useful.

Flow Rate·Water Quality Characteristics of Tributaries and a Grouping Method for Tributary Management in Nakdong River (낙동강 지류·지천의 유량·수질 특성 및 하천관리를 위한 등급화 방안 연구)

  • Na, Seungmin;Lim, Tae Hyen;Lee, Jae Yun;Kwon, Heongak;Cheon, Se Uk
    • Journal of Wetlands Research
    • /
    • v.17 no.4
    • /
    • pp.380-390
    • /
    • 2015
  • In this study, the major 38 tributaries in Nakdong River were monitored for flow rate and water quality in order to understand the characteristics of the watershed and to find improvement plan. The flow rate and water quality for each target tributary were evaluated based on the monitoring data in 2013~2014 using a statistical package SPSS-22.0. In addition, the tributary grouping method was conducted using a $BOD_5$ concentration/flowrate and TP concentration/flowrate monitoring data. The average values of $BOD_5$, $COD_{Mn}$, TP and TOC concentrations in Gumicheon, Gyeonghocheon, Jincheoncheon, Gisegokcheon, Yonghacheon and Yonghocheon located at Nakdong Waegwan and Nakdong Goryung watershed were high and in the grade of III or IV (5~8 mg/L). The Pearson correlation coefficients of TOC with $BOD_5$, $COD_{Mn}$, and TP were greater (r=0.8, p<0.01) than those of the other water quality parameters (12 species). The tributaries with high values of water quality parameters ($BOD_5$ > 3.0 mg/L, TP > 0.1 mg/L) and flowrate (Q > $0.1m^3/sec$) were selected for improving water quality according to the stream grouping method. Five tributaries (Gumicheon, Gisegokcheon, Yonghacheon, Yeongsancheon, Mijeoncheon and Yonghocheon) were classified as Group I, which require polices and plans for water quality improvement.

Evaluation of Water Quality for the Han River Tributaries Using Multivariate Analysis (다변량 통계 분석기법을 이용한 한강수계 지천의 수질 평가)

  • Kim, Yo-Yong;Lee, Si-Jin
    • Journal of Korean Society of Environmental Engineers
    • /
    • v.33 no.7
    • /
    • pp.501-510
    • /
    • 2011
  • In this study, water pollution sources of 14 major tributaries of Han river and characteristics of water quality for each target streams were evaluated based on water quality data in 2007.1-2009.12 (14 data sets) using a statistical package, SPSS-17.0. Cluster analysis over time and space for each stream resulted in 4 groups for the spatial variations in which type and density of pollution sources in the basins showed the greatest impact on grouping. Moreover, cluster analysis for the time variation in which rainfall, temperature and eutrophication were shown to contribute to the clustering, produced 2 groups, from summer to fall (July-Oct.) and from winter to early summer (Nov.-June). Four factors were found as responsible for the data structure explaining 71-90% of the total variance of the data set depending on the streams and they were organic matter, nutrients, bacterial contamination. Factor analysis showed main factors (water pollutants) changed according to the season with different pattern for each stream. This study demonstrated that water quality of each stream could produce useful outcomes when factor and pollution source of basin were evaluated together.

Processing of Sensor Data Stream for OSGi Frameworks (OSGi를 위한 실시간 센서 데이터스트림 처리 방법)

  • Cha, Ji-Yun;Byun, Yung-Cheol;Lee, Dong-Cheal
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.13 no.5
    • /
    • pp.1014-1021
    • /
    • 2009
  • In an environment of home network where a number of technologies including heterogeneous hardware platforms, networking and protocols, middleware systems, and etc, exist, OSGi provides a platform for deployment and sharing of services managed in hardware and guarantees compatibility among applications. However, only simple control and processing of event data are considered in a home network using OSGi, and the consideration about real time processing of data stream generated by sensors is not enough. Therefore, researches allowing users to effectively develop OSGi applications by using various kinds of sensors generating data streams in the home network environment using OSGi are needed. In this paper, we propose an effective method of processing various types of real time data streams supplied to OSGi applications, including filtering, grouping, and counting, etc.

Spatio-temporal Query Clustering: A Data Cubing Approach (시공간 질의 클러스터링: 데이터 큐빙 기법)

  • Chen, Xiangrui;Baek, Sung-Ha;Bae, Hae-Young
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2009.11a
    • /
    • pp.287-288
    • /
    • 2009
  • Multi-query optimization (MQO) is a critical research issue in the real-time data stream management system (DSMS). We propose to address this problem in the ubiquitous GIS (u-GIS) environment, focusing on grouping 'similar' spatio-temporal queries incrementally into N clusters so that they can be processed virtually as N queries. By minimizing N, the overlaps in the data requirements of the raw queries can be avoided, which implies the reducing of the total disk I/O cost. In this paper, we define the spatio-temporal query clustering problem and give a data cubing approach (Q-cube), which is expected to be implemented in the cloud computing paradigm.

Use of Tributary Water Quality and Flowrate Monitoring Data for Effective Implementation of TMDL (수질오염총량관리제의 효율적인 시행을 위한 지류하천 수질.유량모니터링 자료의 활용)

  • Kim, Young-Il;Jeong, Woo-Hyeok;Kim, Hong-Su;Yi, Sang-Jin
    • Journal of Korean Society of Environmental Engineers
    • /
    • v.34 no.2
    • /
    • pp.119-125
    • /
    • 2012
  • The tributary water quality and flowrate monitoring result, which is fundamental data for the establishment of the water environmental policy, is used as very important data for the implementation of TMDL. This study introduced how to use the tributary water quality and flowrate monitoring data for the analysis of the watersheds, the satisfactory assessment of water quality standards in the watersheds, the selection of watersheds for the establishment of the implementation plan, and the selection of the tributary catchments for improving the water quality using a stream grouping method. According to the analytical results of tributary catchment using water quality and flowrate monitoring data of thirty-seven tributaries in the Geum-River watershed at Chungcheongnam-do, the value of flowrate in the tributaries, which is located in the middle-lower Geum-River watershed, was greater than the other areas and the concentration of the water pollutants regardless of water quality parameters in the tributaries at Nonsancheon catchment was relatively higher than the other areas. The problems, which have the determination of satisfaction of water quality standards and selection of target watersheds for establishment of the implementation plan regardless of the water quality of tributary in the watershed due to the water quality and flowrate monitoring results of the main river, were improved use of the results of tributary water quality and flowrate monitoring. Also, the tributary catchments for improving the water quality, according to stream grouping method based on the results of tributary water quality and flowrate monitoring, were selected. In the Geum-River watershed at Chungcheongnam-do, the tributary in the Nonsancheon, Byeongcheoncheon, Seokseongcheon, Jocheon catchments, which has a large flow and a high concentration of water pollutants, should be preferentially selected for improving the water quality of the tributary in accordance with the reduction of the source of pollution.

Temporal and Spatial Analysis of Flowrate and Water Quality of Major Tributaries for Implementation of TMDL in Sapgyo-reservoir Watershed at Chungcheongnam-do (충청남도 삽교호수계 수질오염총량관리제 시행을 위한 주요하천 유량 및 수질의 시.공간적 특성 분석)

  • Park, Sang-Hyun;Moon, Eun-Ho;Cho, Byung-Wook;Choi, Jeong-Ho;Jeong, Woo-Hyeok;Kim, Hong-Su;Yi, Sang-Jin;Kim, Young-Il
    • Journal of Korean Society on Water Environment
    • /
    • v.29 no.1
    • /
    • pp.107-113
    • /
    • 2013
  • The major tributaries in Sapgyo-reservoir watershed at Chungcheongnam-do were monitored for flowrate and water quality in order to analyze the characteristics of watershed and to prepare for implementation of total maximum daily load (TMDL). According to the analytical results of flowrate and water quality monitoring data of sixteen tributaries, the tributaries with the value of flowrate over $0.5m^3/s$ were 62.5% among the monitored tributaries and the value of flowrate in the Cheonancheon, Namwoncheon, Shinyangcheon except Gokgyocheon, Muhancheon, Sapgyocheon was relatively greater than the other tributaries. However, 37.5% of the tributaries were exceeded the water quality standards of Sapgyocheon sub-basin ($BOD_5$ 5 mg/L and/or below) and the concentration of water pollutants regardless of water quality parameters in Cheonancheon, Maegokcheon, Oncheoncheon including Gokgyocheon located in Gokgyocheon catchment were relatively higher than the other tributaries. The tributaries for improving the water quality, according to stream grouping method based on the results of flowrate and water quality monitoring data, were selected. In the Sapgyo-reservoir watershed, the tributaries for improving water quality, which has a large flowrate and a high concentration of water pollutants, were selected at Cheonancheon, Gokgyocheon, Maegokcheon, Namwoncheon, Oncheoncheon. The various water quality improving plans for those tributaries, in accordance with the reduction of point source pollution by population and livestock, should be established and implemented.

Concurrent Channel Time Allocation for Resource Management in WPANs

  • Park, Hyunhee;Piamrat, Kandaraj;Singh, Kamal Deep
    • Journal of information and communication convergence engineering
    • /
    • v.12 no.2
    • /
    • pp.109-115
    • /
    • 2014
  • This paper presents a concurrent channel time allocation scheme used in the reservation period for concurrent transmissions in 60-GHz wireless personal area networks (WPANs). To this end, the proposed resource allocation scheme includes an efficient method for creating a concurrent transmission group by using a table that indicates whether individual streams experience interference from other streams or not. The coordinator device calculates the number of streams that can be concurrently transmitted with each stream and groups them together on the basis of the calculation result. Then, the coordinator device allocates resources to each group such that the streams belonging to the same group can transmit data concurrently. Therefore, when the piconet coordinator (PNC) allocates the channel time to the individual groups, it should allow for maximizing the overall capacity. The performance evaluation result demonstrates that the proposed scheme outperforms the random grouping scheme in terms of the overall capacity when the beamwidth is $30^{\circ}C$ and the radiation efficiency is 0.9.