• Title/Summary/Keyword: Stream Processing System

Search Result 293, Processing Time 0.026 seconds

Development of Information System based on GIS for Analyzing Basin-Wide Pollutant Washoff (유역오염원 수질거동해석을 위한 GIS기반 정보시스템 개발)

  • Park, Dae-Hee;Ha, Sung-Ryong
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.9 no.4
    • /
    • pp.34-44
    • /
    • 2006
  • Simulation models allow researchers to model large hydrological catchment for comprehensive management of the water resources and explication of the diffuse pollution processes, such as land-use changes by development plan of the region. Recently, there have been reported many researches that examine water body quality using Geographic Information System (GIS) and dynamic watershed models such as AGNPS, HSPF, SWAT that necessitate handling large amounts of data. The aim of this study is to develop a watershed based water quality estimation system for the impact assessment on stream water quality. KBASIN-HSPF, proposed in this study, provides easy data compiling for HSPF by facilitating the setup and simulation process. It also assists the spatial interpretation of point and non-point pollutant information and thiessen rainfall creation and pre and post processing for large environmental data An integration methodology of GIS and water quality model for the preprocessing geo-morphologic data was designed by coupling the data model KBASIN-HSPF interface comprises four modules: registration and modification of basic environmental information, watershed delineation generator, watershed geo-morphologic index calculator and model input file processor. KBASIN-HSPF was applied to simulate the water quality impact by variation of subbasin pollution discharge structure.

  • PDF

An Analysis of IT Trends Using Tweet Data (트윗 데이터를 활용한 IT 트렌드 분석)

  • Yi, Jin Baek;Lee, Choong Kwon;Cha, Kyung Jin
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.1
    • /
    • pp.143-159
    • /
    • 2015
  • Predicting IT trends has been a long and important subject for information systems research. IT trend prediction makes it possible to acknowledge emerging eras of innovation and allocate budgets to prepare against rapidly changing technological trends. Towards the end of each year, various domestic and global organizations predict and announce IT trends for the following year. For example, Gartner Predicts 10 top IT trend during the next year, and these predictions affect IT and industry leaders and organization's basic assumptions about technology and the future of IT, but the accuracy of these reports are difficult to verify. Social media data can be useful tool to verify the accuracy. As social media services have gained in popularity, it is used in a variety of ways, from posting about personal daily life to keeping up to date with news and trends. In the recent years, rates of social media activity in Korea have reached unprecedented levels. Hundreds of millions of users now participate in online social networks and communicate with colleague and friends their opinions and thoughts. In particular, Twitter is currently the major micro blog service, it has an important function named 'tweets' which is to report their current thoughts and actions, comments on news and engage in discussions. For an analysis on IT trends, we chose Tweet data because not only it produces massive unstructured textual data in real time but also it serves as an influential channel for opinion leading on technology. Previous studies found that the tweet data provides useful information and detects the trend of society effectively, these studies also identifies that Twitter can track the issue faster than the other media, newspapers. Therefore, this study investigates how frequently the predicted IT trends for the following year announced by public organizations are mentioned on social network services like Twitter. IT trend predictions for 2013, announced near the end of 2012 from two domestic organizations, the National IT Industry Promotion Agency (NIPA) and the National Information Society Agency (NIA), were used as a basis for this research. The present study analyzes the Twitter data generated from Seoul (Korea) compared with the predictions of the two organizations to analyze the differences. Thus, Twitter data analysis requires various natural language processing techniques, including the removal of stop words, and noun extraction for processing various unrefined forms of unstructured data. To overcome these challenges, we used SAS IRS (Information Retrieval Studio) developed by SAS to capture the trend in real-time processing big stream datasets of Twitter. The system offers a framework for crawling, normalizing, analyzing, indexing and searching tweet data. As a result, we have crawled the entire Twitter sphere in Seoul area and obtained 21,589 tweets in 2013 to review how frequently the IT trend topics announced by the two organizations were mentioned by the people in Seoul. The results shows that most IT trend predicted by NIPA and NIA were all frequently mentioned in Twitter except some topics such as 'new types of security threat', 'green IT', 'next generation semiconductor' since these topics non generalized compound words so they can be mentioned in Twitter with other words. To answer whether the IT trend tweets from Korea is related to the following year's IT trends in real world, we compared Twitter's trending topics with those in Nara Market, Korea's online e-Procurement system which is a nationwide web-based procurement system, dealing with whole procurement process of all public organizations in Korea. The correlation analysis show that Tweet frequencies on IT trending topics predicted by NIPA and NIA are significantly correlated with frequencies on IT topics mentioned in project announcements by Nara market in 2012 and 2013. The main contribution of our research can be found in the following aspects: i) the IT topic predictions announced by NIPA and NIA can provide an effective guideline to IT professionals and researchers in Korea who are looking for verified IT topic trends in the following topic, ii) researchers can use Twitter to get some useful ideas to detect and predict dynamic trends of technological and social issues.

Design and Implementation of a Transparent Security Infrastructure using Filter and Transformer (필터와 트랜스포머를 이용한 투명한 보안기반의 설계 및 구현)

  • Kim, Yong-Min;Lee, Do-Heon;Noh, Bong-Nam;Choi, Rak-Man;Ine, So-Ran
    • The Transactions of the Korea Information Processing Society
    • /
    • v.5 no.4
    • /
    • pp.975-983
    • /
    • 1998
  • In these days, information communication systems are based on both open distributed computing technologies and object-oriented techniques like inheritance, encapsulation and object reuse to support various system configuration and application. As information systems are interconnected through unsecure networks, the need for the secure information exchange is more critical than before. In this paper, we have designed and implemented a transparent CORBA-basce Security infrastructure with authentication, security context association, access control and security information management to support a secure applications in distributed object environment. SESAME Ver. 4 was adopted as an external security service to manage user privilege attributes and to distribute keys for data encryption, decryption and integrity. Using filter and transformer with an interface to Object Request Broker, it provides a transparent security service to applications. The filter objects are special classes that allow additional parameters to be inserted into messages before they are sent and removed just after they are received. The transformer objects are special classes that allow direct access to the byte stream of every messages for encryption and decryption before it is sent and just after it is received. This study is to implement the access control interceptor(ACI) and the secure invocation interceptor(SII) of secure ORB defined in CORBA using filter and transformer.

  • PDF

An Architecture for Managing Faulty Sensing Data on Low Cost Sensing Devices over Manufacturing Equipments (전문 설비의 이상신호 처리를 위한 저비용 관제 시스템 구축)

  • Chae, Yuna;Kim, Changi;Ko, Haram;Kim, Woongsup
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.7 no.3
    • /
    • pp.113-120
    • /
    • 2018
  • In this study, we proposed a monitoring system for identifying and handling faulty sensing stream data on manufacturing equipments where low-cost sensors can be safely used. Low cost sensors will lessen the cost of implementing distributed monitoring system, but suffer from sensor noises and inaccurate sensed data. Therefore, a distributed monitoring system with low cost sensors should identify faulty signal data as either of sensor fault or machine fault, and filter out faulty signals from sensing fault. To this end, we adopted a fourier transform based diagnostic approach mixed with a weighed moving averaging method, in order to identify faulty signals. We measured how effective our approach is and found out our approach can filter out one-third faulty signals from our experimental environment. In addition, we attached wireless communication modules to reduce sensor and network installation cost. To handle massive sensor data efficiently, we employed unstructured data format with NoSQL based database.

Development of a GIS-Based Basin Water Balance Analysis Model (GIS 기반의 유역물수지 분석모형 개발)

  • Hwang, Eui-Ho;Kim, Kye-Hyun;Park, Jin-Hyeog;Lee, Geun-Sang
    • Journal of the Korean Association of Geographic Information Studies
    • /
    • v.7 no.4
    • /
    • pp.34-45
    • /
    • 2004
  • Existing Semangeum's water balance analysis simplifies whole basin to single basin and achieved volume of effluence that produce by Kajiyama way to foundation. But Semangeum is complicated and various rice-wine strainer supply system. And there is difficulty to apply as elastic when water balance element is changed at free point. Divided to unit possession station for suitable water balance analysis model application to Semangeum in this study. And developed basin water balance model of GIS base that can do details analysis is bite about development and transfer of an appropriation in the budget of basin water resources. Achieved study including abstraction and concept design that use UML (unified modeling language) diagram for details analysis, stream network composition for rice-wine strainer supply system application, preprocessing of GIS base and postprocessing module development, model revision and verification etc. Support of this water balance analysis model is available to establish efficient water resources administration plan through outward flow process analysis of water resources. And support is considered to be possible in more convenient and, reasonable water resources administration way establishment by minimizing manual processing in systematic water resources government official to user and support diversified analysis system.

  • PDF

Effective Load Shedding for Multi-Way windowed Joins Based on the Arrival Order of Tuples on Data Streams (다중 윈도우 조인을 위한 튜플의 도착 순서에 기반한 효과적인 부하 감소 기법)

  • Kwon, Tae-Hyung;Lee, Ki-Yong;Son, Jin-Hyun;Kim, Myoung-Ho
    • Journal of KIISE:Databases
    • /
    • v.37 no.1
    • /
    • pp.1-11
    • /
    • 2010
  • Recently, there has been a growing interest in the processing of continuous queries over multiple data streams. When the arrival rates of tuples exceed the memory capacity of the system, a load shedding technique is used to avoid the system becoming overloaded by dropping some subset of input tuples. In this paper, we propose an effective load shedding algorithm for multi-way windowed joins over multiple data streams. Most previous load shedding algorithms estimate the productivity of each tuple, i.e., the number of join output tuples produced by the tuple, based on its "join attribute value" and drop tuples with the lowest productivity. However, the productivity of a tuple cannot be accurately estimated from its join attribute value when the join attribute values are unique and do not repeat, or the distribution of the join attribute values changes over time. For these cases, we estimate the productivity of a tuple based on its "arrival order" on data streams, rather than its join attribute value. The proposed method can effectively estimate the productivity of a tuple even when the productivity of a tuple cannot be accurately estimated from its join attribute value. Through extensive experiments and analysis, we show that our proposed method outperforms the previous methods in terms of effectiveness and efficiency.

DCT-domain MPEG-2/H.264 Video Transcoder System Architecture for DMB Services (DMB 서비스를 위한 DCT 기반 MPEG-2/H.264 비디오 트랜스코더 시스템 구조)

  • Lee Joo-Kyong;Kwon Soon-Young;Park Seong-Ho;Kim Young-Ju;Chung Ki-Dong
    • The KIPS Transactions:PartB
    • /
    • v.12B no.6 s.102
    • /
    • pp.637-646
    • /
    • 2005
  • Most of the multimedia contents for DBM services art provided as MPEG-2 bit streams. However, they have to be transcoded to H.264 bit streams for practical services because the standard video codec for DMB is H.264. The existing transcoder architecture is Cascaded Pixel-Domain Transcoding Architecture, which consists of the MPEG-2 dacoding phase and the H.264 encoding phase. This architecture can be easily implemented using MPEG-2 decoder and H.264 encoder without source modifying. However. It has disadvantages in transcoding time and DCT-mismatch problem. In this paper, we propose two kinds of transcoder architecture, DCT-OPEN and DCT-CLOSED, to complement the CPDT architecture. Although DCT-OPEN has lower PSNR than CPDT due to drift problem, it is efficient for real-time transcoding. On the contrary, the DCT-CLOSED architecture has the advantage of PSNR over CPDT at the cost of transcoding time.

Network Adaptive Quality of Service Method in Client/Server-based Streaming Systems (클라이언트/서버 기반 스트리밍 시스템에서의 네트워크 적응형 QoS 기법)

  • Zhung, Yon-il;Lee, Jung-chan;Lee, Sung-young
    • The KIPS Transactions:PartA
    • /
    • v.10A no.6
    • /
    • pp.691-700
    • /
    • 2003
  • Due to the fast development of wire&wireless internet and computer hardware, more and more internet services are being developed, such as Internet broadcast, VoD (Video On Demand), etc. So QoS (Qualify of Service) is essentially needed to guarantee the quality of these services. Traditional Internet is Best-Effort service in which all packets are transported in FIFO (First In First Out) style. However, FIFO is not suitable to guarantee the quality of some services, so more research in QoS router and QoS protocol are needed. Researched QoS router and protocol are high cost and inefficient because the existing infra is not used. To solve this problem, a new QoS control method, named Network Adaptive QoS, is introduced and applied to client/server-based streaming systems. Based on network bandwidth monitoring mechanism, network adaptive QoS control method can be used in wire&wireless networks to support QoS in real-time streaming system. In order to reduce application cost, the existing streaming service is used in NAQoS. A new module is integrated into the existing server and client. So the router and network line are not changed. By simulation in heavy traffic network conditions, we proved that stream cannot be seamless without network adaptive QoS method.

The Implementation of Multi-Channel Audio Codec for Real-Time operation (실시간 처리를 위한 멀티채널 오디오 코덱의 구현)

  • Hong, Jin-Woo
    • The Journal of the Acoustical Society of Korea
    • /
    • v.14 no.2E
    • /
    • pp.91-97
    • /
    • 1995
  • This paper describes the implementation of a multi-channel audio codec for HETV. This codec has the features of the 3/2-stereo plus low frequency enhancement, downward compatibility with the smaller number of channels, backward compatibility with the existing 2/0-stereo system(MPEG-1 audio), and multilingual capability. The encoder of this codec consists of 6-channel analog audio input part with the sampling rate of 48 kHz, 4-channel digital audio input part and three TMS320C40 /DSPs. The encoder implements multi-channel audio compression using a human perceptual psychoacoustic model, and has the bit rate reduction to 384 kbit/s without impairment of subjective quality. The decoder consists of 6-channel analog audio output part, 4-channel digital audio output part, and two TMS320C40 DSPs for a decoding procedure. The decoder analyzes the bit stream received with bit rate of 384 kbit/s from the encoder and reproduces the multi-channel audio signals for analog and digital outputs. The multi-processing of this audio codec using multiple DSPs is ensured by high speed transfer of date between DSPs through coordinating communication port activities with DMA coprocessors. Finally, some technical considerations are suggested to realize the problem of real-time operation, which are found out through the implementation of this codec using the MPEG-2 layer II sudio coding algorithm and the use of the hardware architecture with commercial multiple DSPs.

  • PDF

Mobile Presentation using Transcoding Method of Region of Interest (관심 영역의 트랜스코딩 기법을 이용한 모바일 프리젠테이션)

  • Seo, Jung-Hee;Park, Hung-Bog
    • The KIPS Transactions:PartC
    • /
    • v.17C no.2
    • /
    • pp.197-204
    • /
    • 2010
  • An effective integration of web-based learning environment and mobile device technology is considered as a new challenge to the developers. The screen size, however, of the mobile device is too small, and its performance is too inferior. Due to the foregoing limit of mobile technology, displaying bulk data on the mobile screen, such as a cyber lecture accompanied with real-time image transmission on the web, raises a lot of problems. Users have difficulty in recognizing learning contents exactly by means of a mobile device, and continuous transmission of video stream with bulky information to the mobile device arouses a lot of load for the mobile system. Thus, an application which is developed to be applied in PC is improper to be used for the mobile device as it is, a player which is fitting for the mobile device should be developed. Accordingly, this paper suggests mobile presentation using transcoding techniques of the field concerned. To display continuous video frames of learning image, such as a cyber lecture or remote lecture, by means of a mobile device, the performance difference between high-resolution digital image and mobile device should be surmounted. As the transcoding techniques to settle the performance difference causes damage of image quality, high-quality image may be guaranteed by application of trial and error between transcoding and selected learning resources.