• Title/Summary/Keyword: kafka

Search Result 34, Processing Time 0.017 seconds

A System Design for Real-Time Monitoring of Patient Waiting Time based on Open-Source Platform (오픈소스 플랫폼 기반의 실시간 환자 대기시간 모니터링 시스템 설계)

  • Ryu, Wooseok
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.22 no.4
    • /
    • pp.575-580
    • /
    • 2018
  • This paper discusses system for real-time monitoring of patient waiting time in hospitals based on open-source platform. It is necessary to make use of open-source projects to develop a high-performance stream processing system, which analyzes and processes stream data in real time, with less cost. The Hadoop ecosystem is a well-known big data processing platform consisting of numerous open-source subprojects. This paper first defines several requirements for the monitoring system, and selects a few projects from the Hadoop ecosystem that are suited to meet the requirements. Then, the paper proposes system architecture and a detailed module design using Apache Spark, Apache Kafka, and so on. The proposed system can reduce development costs by using open-source projects and by acquiring data from legacy hospital information system. High-performance and fault-tolerance of the system can also be achieved through distributed processing.

A Study on the Outside of Discourse from the Views of Foucault and Bakhtin (푸코와 바흐친을 통해 바라본 담론의 바깥)

  • Jo, Su-gyeong
    • Journal of Korean Philosophical Society
    • /
    • v.117
    • /
    • pp.327-354
    • /
    • 2011
  • This study has a key assumption that 'characteristics of discouse can be found in not its inside, but its outside'. The prism through which we can see those characteristics was provided by Foucault and Bakhtin who were introduced in the study. As an effort co probe the outside of discourse, the study is briefed 'the three attributes of discourse' that were suggested by Foucault First, discourse has the principle of selection and exclusion, which is based on power relations. Second, discourse is not transparent at all since it is always offensive towards other discourses and defensive against selected ones Third, discourse which is naturally accepted because of its dailiness had a deep structure secretly hidden in it. Based on the above attributes, Foucault and Bakhtin paid their attention to the outside of discourse. Specifically, they considered discourse fundamentally and went beyond it, and reflected the procedures of discourse. This study focused on 'Socrates', something common in the two scholars' works with discourse. In dealing with discourse, Bakhtin started with 'Socratic dialogue' that is based on the dialogic nature of human thoughts which purse the truth. For Foucault, it was Socrates who had the 'courage to cell the truth' and practiced 'self-consideration'. According to Foucault, the ethics of self-practice originated from the philosopher. The ethics is neither the precise representation of individual life that is withdrawn towards the inner self, nor the skills of happiness. It is just relational and cross-sectional. For a better understanding, this study pointed out that Kafka created a variety of 'dialogic voices' focusing on the outside of discourse. Dialogues found in his writings are 'interminable dialogues' that truly 'communicate with different times and different spaces'. For example, his novel, 'Der Prozess' opens the possibility of discussing in various ways the court which is look beyond conventions and extraordinary. Kafka's novels have a structure that their starting point found at the introduction reappears at the termination, presenting multi-vocal dialogues.

Development of Information Technology Infrastructures through Construction of Big Data Platform for Road Driving Environment Analysis (도로 주행환경 분석을 위한 빅데이터 플랫폼 구축 정보기술 인프라 개발)

  • Jung, In-taek;Chong, Kyu-soo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.3
    • /
    • pp.669-678
    • /
    • 2018
  • This study developed information technology infrastructures for building a driving environment analysis platform using various big data, such as vehicle sensing data, public data, etc. First, a small platform server with a parallel structure for big data distribution processing was developed with H/W technology. Next, programs for big data collection/storage, processing/analysis, and information visualization were developed with S/W technology. The collection S/W was developed as a collection interface using Kafka, Flume, and Sqoop. The storage S/W was developed to be divided into a Hadoop distributed file system and Cassandra DB according to the utilization of data. Processing S/W was developed for spatial unit matching and time interval interpolation/aggregation of the collected data by applying the grid index method. An analysis S/W was developed as an analytical tool based on the Zeppelin notebook for the application and evaluation of a development algorithm. Finally, Information Visualization S/W was developed as a Web GIS engine program for providing various driving environment information and visualization. As a result of the performance evaluation, the number of executors, the optimal memory capacity, and number of cores for the development server were derived, and the computation performance was superior to that of the other cloud computing.

Design and Implemention of Real-time web Crawling distributed monitoring system (실시간 웹 크롤링 분산 모니터링 시스템 설계 및 구현)

  • Kim, Yeong-A;Kim, Gea-Hee;Kim, Hyun-Ju;Kim, Chang-Geun
    • Journal of Convergence for Information Technology
    • /
    • v.9 no.1
    • /
    • pp.45-53
    • /
    • 2019
  • We face problems from excessive information served with websites in this rapidly changing information era. We find little information useful and much useless and spend a lot of time to select information needed. Many websites including search engines use web crawling in order to make data updated. Web crawling is usually used to generate copies of all the pages of visited sites. Search engines index the pages for faster searching. With regard to data collection for wholesale and order information changing in realtime, the keyword-oriented web data collection is not adequate. The alternative for selective collection of web information in realtime has not been suggested. In this paper, we propose a method of collecting information of restricted web sites by using Web crawling distributed monitoring system (R-WCMS) and estimating collection time through detailed analysis of data and storing them in parallel system. Experimental results show that web site information retrieval is applied to the proposed model, reducing the time of 15-17%.