• Title/Summary/Keyword: Stream Processing System

Search Result 294, Processing Time 0.026 seconds

A Study on the Data Collection Methods based Hadoop Distributed Environment (하둡 분산 환경 기반의 데이터 수집 기법 연구)

  • Jin, Go-Whan
    • Journal of the Korea Convergence Society
    • /
    • v.7 no.5
    • /
    • pp.1-6
    • /
    • 2016
  • Many studies have been carried out for the development of big data utilization and analysis technology recently. There is a tendency that government agencies and companies to introduce a Hadoop of a processing platform for analyzing big data is increasing gradually. Increased interest with respect to the processing and analysis of these big data collection technology of data has become a major issue in parallel to it. However, study of the collection technology as compared to the study of data analysis techniques, it is insignificant situation. Therefore, in this paper, to build on the Hadoop cluster is a big data analysis platform, through the Apache sqoop, stylized from relational databases, to collect the data. In addition, to provide a sensor through the Apache flume, a system to collect on the basis of the data file of the Web application, the non-structured data such as log files to stream. The collection of data through these convergence would be able to utilize as a basic material of big data analysis.

Design and Implementation of the Notification System using Event-Profile Filtering (이벤트-프로파일 여과를 이용한 통지시스템의 설계 및 구현)

  • Ban, Chae-Hoon
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2010.10a
    • /
    • pp.129-132
    • /
    • 2010
  • Users can obtain useful information from large of data because of development of internet. Since these data rise continuously like stream data, it is required to extract the appropriate information efficiently for the user's needs. In the traditional scheme, they are acquired by processing the user queries after the occurred data are stored at a database. However, it is inefficient to process the user queries over the large volume of continuous data by using the traditional scheme. In this paper, we propose the Event-Profile Model to define the data occurrence on the internet as the events and the user's requirements as the profiles. We also propose and implement the filtering scheme to process the events and the profiles efficiently. We evaluate the performance of the proposed scheme and our experiments show that the new scheme outperforms the other on various dataset.

  • PDF

Design and Implementation of the Notification System based on the Event-Profile Model (이벤트-프로파일 모델을 기반으로 한 통지 시스템의 설계 및 구현)

  • Ban, Chae-Hoon;Kim, Dong-Hyun
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.15 no.8
    • /
    • pp.1750-1755
    • /
    • 2011
  • Recently, it is possible for users to acquire necessary data easily as the various schemes of the searching information are developed. Since these data rise continuously like stream data, it is required to extract the appropriate data for the user's needs from the mass data on the internet. In the traditional scheme, they are acquired by processing the user queries after the occurred data are stored at a database. However, it is inefficient to process the user queries over the large volume of continuous data by using the traditional scheme. In this paper, we propose the Event-Profile Model to define the data occurrence on the internet as the events and the user's requirements as the profiles. We also propose and implement the filtering scheme to process the events and the profiles efficiently. We evaluate the performance of the proposed scheme and our experiments show that the new scheme outperforms the other on various dataset.

Combustion Characteristics of Premixed Burner for Fuel Reformer (개질기용 예혼합 연소장치의 연소특성 연구)

  • Lee, Pil-Hyong;Lee, Jae-Young;Han, Sang-Seok;Park, Chang-Soo;Hwang, Sang-Soon
    • Proceedings of the KSME Conference
    • /
    • 2008.11b
    • /
    • pp.2181-2185
    • /
    • 2008
  • Fuel processing systems which convert HC fuel into $H_2$ rich gas (such as stream reforming, partial oxidation, auto-thermal reforming) need high temperature environment($600-1000^{\circ}C$). Generally, anode-off gas or mixture of anode-off gas and LNG is used as input gas of fuel reformer. In order to make efficient and low emission burner system for fuel reformer, it is necessary to elucidate the combustion and emission characteristic of fuel reformer burner. The purpose of this study is to develop a porous premixed flat ceramic burner that can be used for 1-5kW fuel cell reformer. Ceramic burner experiments using natural gas, hydrogen gas, anode off gas were carried out respectively to investigate the flame characteristics by heating capacity and equivalence ratio. Results show that the stable flat flames can be established for natural gas, hydrogen gas, anode off gas and mixture of natural & anode off gas as reformer fuel. For all of fuels, their burning velocities become smaller as the equivalence ratio goes to the lean mixture ratio, and a lift-off occurs at lean limit. Flame length in hydrogen and anode off gas became longer with increasing the heat capacity.

  • PDF

The Trend of Cigarette Design and Tobacco Flavor System Development

  • Wu, Jimmy Z.
    • Journal of the Korean Society of Tobacco Science
    • /
    • v.24 no.1
    • /
    • pp.67-73
    • /
    • 2002
  • In light of addressing consumer health concern, coping with anti-tobacco movement, and promoting new product, tobacco industry is actively pursuing to make a new generation of cigarettes with low tar and nicotine deliveries, and less harmful substances. Low tar and low nicotine cigarettes increases their market shares dramatically world wide, especially in KT&G, multinational tobacco companies, EU countries, even in China regulated by CNTC to set up yearly target to lower tar and nicotine deliveries. On the other hand, to design a new cigarette with reduced harmful substances begins to gain speed. The "modified Hoffmann list" publishes thirty plus substances in tobacco leaf and main smoke stream, which is the prime suspect causing health problems. Various ways and means are developed to reduce such components including new tobacco breeds, new curing method, tobacco leaf treatment before processing, selected filtration system, innovated casing system to reduce free radicals, as well as some non conventional cigarette products. In TSRC held this year, the main topic is related to reduce tobacco specific nitrosamines in tobacco leaf. The new generation of cigarette is in the horizon. It still needs a lot help to produce commercial products with satisfied taste and aroma characters. The flavor industry is not regulated by many governments demanding which ingredients might or might not be for tobacco use. However, most of the cigarette companies self impose a list of ingredients to guide flavor suppliers to design flavors. Unfortunately, the number of ingredients in those lists is getting shorter every year. It is understandable that the health is not the only reason. Some cigarette companies are playing safe to protect the company from potential lawsuit, while others are just copying from their competitors. Moreover, it is obvious that it needs more assistance from casings and flavors to design new generation of cigarettes with missing certain flavor components in tobacco leaf and main smoke stream. These flavor components are either non-existed or at lower level at new form of cured tobacco leaf or filtered in the main smoke stream along with reduced harmful substances. The use of carbon filters and other selected filtration system poses another tough task for flavor system design. Specific flavor components are missing from the smoke analysis data, which brings a notion of "carbon taste" and "dryness" of mouth feel. It is ever more demanded by cigarette industry to flavor suppliers to produce flavors as body enhancer, tobacco notes, salivating agents, harshness reducer, and various of aromatic notes provided they are safe to use. Another trend is that water based flavor or flavor with reduced ethanol as solvent is gaining popularity. It is preferred by some cigarette companies that the flavor is compounded with all natural ingredients or all ingredients should he GMO free. The new generation of cigarettes demands many ways of new thinking process. It is also vital for tobacco industry. It reflects the real needs for the consumers that the cigarette product should be safe to use as well as bearing the taste and aroma characters smokers always enjoyed. An effective tobacco flavor system is definitely a part of the equation. The global trend of tobacco industry is like trends of any other industries lead by consumer needs, benefited with new technology availability, affected by the global economy, and subjected for various rules and regulations. Anti-tobacco organizations and media exceptionally scrutinize cigarette, as a legal commercial product. Cigarette is probably the most studied commercial product for its composition, structure, deliveries, effects, as well as its new developmental trend. Therefore, any new trend of cigarette development would be within these boundaries. This paper is trying to point out what it would be like for tobacco industry in the next few yews and what concerns the tobacco industry. It focuses mostly on the efforts to produce safer cigarettes. It is such a vital task for the tobacco industry and its affiliate industries such as cigarette papers, filters, flavors, and other materials. The facts and knowledge presented in this paper might be well known for the public. Some of the comments and predictions are very much personal opinion for a further discussion.

Dangerous Area Prediction Technique for Preventing Disaster based on Outside Sensor Network (실외 센서네트워크 기반 재해방지 시스템을 위한 위험지역 예측기법)

  • Jung, Young-Jin;Kim, Hak-Cheol;Ryu, Keun-Ho
    • The KIPS Transactions:PartD
    • /
    • v.13D no.6 s.109
    • /
    • pp.775-788
    • /
    • 2006
  • Many disaster monitoring systems are constantly studied to prevent disasters such as environmental pollution, the breaking of a tunnel and a building, flooding, storm earthquake according to the progress of wireless telecommunication, the miniaturization of terminal devices, and the spread of sensor network. A disaster monitoring system can extract information of a remote place, process sensor data with rules to recognize disaster situation, and provide work for preventing disaster. However existing monitoring systems are not enough to predict and prevent disaster, because they can only process current sensor data through utilizing simple aggregation function and operators. In this paper, we design and implement a disaster prevention system to predict near future dangerous area through using outside sensor network and spatial Information. The provided prediction technique considers the change of spatial information over time with current sensor data, and indicates the place that could be dangerous in near future. The system can recognize which place would be dangerous and prepare the disaster prevention. Therefore, damage of disaster and cost of recovery would be reduced. The provided disaster prevention system and prediction technique could be applied to various disaster prevention systems and be utilized for preventing disaster and reducing damages.

Implementation of a Secure Wireless LAU System using AP Authentication and Dynamic Key Exchange (AP 인증 및 동적 키 분배를 이용한 안전한 무선랜 시스템 구현)

  • Lee, Jong-Hu;Lee, Myung-Sun;Ryou, Jae-Cheol
    • The KIPS Transactions:PartC
    • /
    • v.11C no.4
    • /
    • pp.497-508
    • /
    • 2004
  • The existing wireless LAN standard IEEE802.11b has many vulnerabilities from security point of view. The authentication mechanisms in IEEE802.11b have many vulnerabilities. As a result to complement the weak of IEEE802.11b authentication, the IEEE802.1x had been developed in the sense of providing strong user authentication with appropriate mechanism. But this mechanism does not perform AP authentication and there are also some weak points. And in confidentiality and message Integrity case, WEP is weak from key stream reuse attack, IV reuse attack and so on. For that reason, in this paper we propose secure wireless LAN system. Our system provides strong user authentication, confidentiality, and message integrity based on existing IEEE802.1x framework and TLS.

Synchronized MP3 Playing System Using XML Extension of MP3 Tag (MP3 태그의 XML 확장을 이용한 동기화된 재생 시스템)

  • Gwak, Mi-Ra;Jo, Dong-Seop
    • The KIPS Transactions:PartB
    • /
    • v.9B no.1
    • /
    • pp.67-76
    • /
    • 2002
  • MP3 audio format has good quality and high compression rate ; therefore, the use of MP3 format increases. The requirement of keeping the extra information such as author and lyrics in MP3 files increases. And the tagging systems designed to meet this requirement are suggested. ID3 vl tag and Lyrics3 v2 tag are two most widely used tagging systems. But ID3 vl tag and Lyrics3 v2 tag are the last things to arrive when the file is being streamed. Therefore, users cannot get the tag information until the entire audio file is downloaded. Moreover information synchronized with audio stream may lose its feature. In this paper, a system searching and playing audio files based on tag information in MP3 files is implemented. This system solves the problem that the tag information is ignored when an MP3 files is played on internet. An audio object is described in an XML document, and timing and synchronization between elements in that In document is provided in HTML+TIME style using XSL.

A Study of Hybrid Cryptosystem Design with the Authentication and Self-Key Generation (인증기능과 자기 키 생성기능을 가진 혼합형 암호시스템 설계에 관한 연구)

  • 이선근;송제호;김태형;김환용
    • Journal of the Institute of Electronics Engineers of Korea SD
    • /
    • v.40 no.9
    • /
    • pp.702-713
    • /
    • 2003
  • The importance of protection for data and information is increasing by the rapid development of information communication and network. And the concern for protecting private information is also growing due to the increasing demand for lots of services by users. Asymmetric cryptosystem is the mainstream in encryption system rather than symmetric cryptosystem by above reasons. But asymmetric cryptosystem is restricted in applying fields by the reason it takes more times to process than symmetric cryptosystem. In this paper, encryption system which executes authentication works of asymmetric cryptosystem by means of symmetric cryptosystem. The proposed cryptosystem uses an algorithms combines that combines block cipherment with stream cipherment and has a high stability in aspect of secret rate by means of transition of key sequence according to the information of plaintext while symmetric/asymmetric cryptosystem conducts encipherment/deciphermeent using a fixed key. Consequently, it is very difficult to crack although unauthenticator acquires the key information. So, the proposed encryption system which has a certification function of asymmetric cryptosystem and a processing time equivalent to symmetric cryptosystems will be highly useful to authorize data or exchange important information.

Video GIS using Virtual World Mapping Technique (가상 세계 매핑기법을 이용한 비디오 GIS)

  • Kim, Sung-Soo;Lee, Seong-Ho;Kim, Kyung-Ok;Lee, Jong-Hun
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.10 no.4 s.22
    • /
    • pp.31-39
    • /
    • 2002
  • In this paper, we propose Media CIS system architecture which provides geographical information of geo-features in video sequences by using 3D geographical database and GPS-related data resulted from 4S-Van. We introduce a novel interoperable geographical data service concept, so-called, Virtual World Mapping (VWA) that can map 3D graphic world with real-world video to provide geographical information. Our proposed method can easily retrieve geographical information and attributes to reconstruct 3D virtual space according to certain frame in video sequences. Our proposed system architecture also has an advantage that can provide geographical information service with video stream without any image processing procedures. In addition to, describing the details of our components, we present a Media GIS web service system by using GeoVideo Server, which performs VWM technique.

  • PDF