• Title/Summary/Keyword: kafka

Search Result 34, Processing Time 0.025 seconds

Safety Autonomous Platform Design with Ensemble AI Models (앙상블 인공지능 모델을 활용한 안전 관리 자율운영 플랫폼 설계)

  • Dongyeop Lee;Daesik Lim;Soojeong Woo;Youngho Moon;Minjeong Kim;Joonwon Lee
    • Journal of Advanced Navigation Technology
    • /
    • v.28 no.1
    • /
    • pp.159-162
    • /
    • 2024
  • This paper proposes a novel safety autonomous platform (SAP) architecture that can automatically and precisely manage on-site safety through ensemble artificial intelligence models generated from video information, worker's biometric information, and the safety rule to estimate the risk index. We practically designed the proposed SAP architecture by the Hadoop ecosystem with Kafka/NiFi, Spark/Hive, Hue, ELK (Elasticsearch, Logstash, Kibana), Ansible, etc., and confirmed that it worked well with safety mobility gateways for providing various safety applications.

Destitution as an Expenditure: Beckett's Literature of Poverty (소모로서의 궁핍: 베케트의 빈궁문학)

  • Park, Ilhyung
    • English & American cultural studies
    • /
    • v.10 no.2
    • /
    • pp.73-97
    • /
    • 2010
  • Representation of destitution may be considered as an expression of a social desire toward forging a bond or solidarity with the impoverished. However, political and ethical demands of the solidarity force the formulaic framework structuring the form of representation to its limits. The thesis aims to examine the responses to such demands within the tradition of modernist literature that can be traced from Charles Baudelaire, Knut Hamsun to Franz Kafka and that somehow culminates with Samuel Beckett, and to analyze how the issue of destitution that weaves through Beckett's works criticizes and inherits such a heritage. Whereas destitution in 19th century Realism is structurally fixed and its potential for change is inherently excluded, for these writers, destitution is no longer the state of rigid reality in which any possibility is limited. It is destitution as an imperative that calls for exploitation of possibilities that can be recuperated from the impoverished condition of destitution. What these writers consistently resist against is destitution that leads to compensation and reward. Since occupying a superior position toward the other as the subject of description or sympathy can be seen as one form of profit or reward, they have persistently pursued absolute solitariness and austere conditions rather than prematurely simulating a sense of solidarity and community. The ultimate goal of destitution as an imperative is to pursue destitution in order to worsen it by identifying and then excluding and expending possessions and assets to a state of penury. This is a paradoxical process that opens up the realm of possibilities of destitution and redefines it as abundance and wealth. Destitution for Beckett as seen in the writers above is the objective of literature. But, what he focuses on is to amplify the shreds of economic world that still remain in a state of poverty and to reveal extreme poverty as a state of odd affluence and to transform it into a pursuit of accumulation and profit. One of his famous axioms, "less is more", contains the essence of such a paradoxical strategy. In a sense, such approach is a twist on the strategy that identifies and uses any remaining potential hidden in destitution as was pursued by other writers. It also expands on the imagination of the destitute described by Hamsun. But Hamsun and Beckett are diametrical opposites. Unlike Hamsun, Beckett does not link imagination with a sense of guilt. Imagination is not intended to overcome the destitute reality nor to culminate in artistic martyrdom as in the case of Kafka's hunger artist. The imagination of the impoverished in Beckett is simply a hilarious game and not an escape that ends in a sense of guilt. This game formulates a "rhetorical question" or derision at the ironical situation where the pursuit of hunger and art as the disinterestedness has been turned into symbolic capital. It is inherently a fundamental critique at the aestheticization of destitution that has been pursued by Modernism. Beckett's efforts at divulging falsehood inherent in non-profit acts such as charity, donation and hospitality are dissections of social fictions in which aestheticization of destitution remains a part of the whole.

Location Tracking and Visualization of Dynamic Objects using CCTV Images (CCTV 영상을 활용한 동적 객체의 위치 추적 및 시각화 방안)

  • Park, Sang-Jin;Cho, Kuk;Im, Junhyuck;Kim, Minchan
    • Journal of Cadastre & Land InformatiX
    • /
    • v.51 no.1
    • /
    • pp.53-65
    • /
    • 2021
  • C-ITS(Cooperative Intelligent Transport System) that pursues traffic safety and convenience uses various sensors to generate traffic information. Therefore, it is necessary to improve the sensor-related technology to increase the efficiency and reliability of the traffic information. Recently, the role of CCTV in collecting video information has become more important due to advances in AI(Artificial Intelligence) technology. In this study, we propose to identify and track dynamic objects(vehicles, people, etc.) in CCTV images, and to analyze and provide information about them in various environments. To this end, we conducted identification and tracking of dynamic objects using the Yolov4 and Deepsort algorithms, establishment of real-time multi-user support servers based on Kafka, defining transformation matrices between images and spatial coordinate systems, and map-based dynamic object visualization. In addition, a positional consistency evaluation was performed to confirm its usefulness. Through the proposed scheme, we confirmed that CCTVs can serve as important sensors to provide relevant information by analyzing road conditions in real time in terms of road infrastructure beyond a simple monitoring role.

Messaging System Analysis for Effective Embedded Tester Log Processing (효과적인 Embedded Tester Log 처리를 위한 Messaging System 분석)

  • Nam, Ki-ahn;Kwon, Oh-young
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2017.05a
    • /
    • pp.645-648
    • /
    • 2017
  • The existing embedded tester used TCP and shared file system for log processing. In addition, the existing processing method was treated as 1-N structure. This method wastes resources of the tester for exception handling. We implemented a log processing message layer that can be distributed by messaging system. And we compare the transmission method using the message layer and the transmission method using TCP and the shared file system. As a result of comparison, transmission using the message layer showed higher transmission bandwidth than TCP. In the CPU usage, the message layer showed lower efficiency than TCP, but showed no significant difference. It can be seen that the log processing using the message layer shows higher efficiency.

  • PDF

Apache Storm based Query Filtering System for Multivariate Data Streams (다변량 데이터 스트림을 위한 아파치 스톰 기반 질의 필터링 시스템)

  • Kim, Youngkuk;Son, Siwoon;Moon, Yang-Sae
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2018.10a
    • /
    • pp.561-564
    • /
    • 2018
  • 최근 빠르게 발생하는 빅데이터 스트림이 다양한 분야에서 활용되고 있다. 이러한 빅데이터 전체를 수집하고 처리하는 것은 매우 비경제적이므로, 데이터 스트림 중 필요한 데이터를 걸러내는 필터링 과정이 필요하다. 본 논문에서는 아파치 스톰(Apache Storm)을 사용하여 데이터 스트림의 질의 필터링 시스템을 구축한다. 스톰은 대용량 데이터 스트림을 처리하기 위한 실시간 분산 병렬 처리 프레임워크이다. 하지만, 스톰은 입력 데이터 구조나 알고리즘 변경 시, 코드의 수정과 재배포, 재시작 등이 필요하다. 따라서, 본 논문에서는 이 같은 문제를 해결하기 위해 아파치 카프카(Apache Kafka)를 사용하여 데이터 수집 모듈과 스톰의 처리 모듈을 분리함으로써 시스템의 가용성을 크게 높인다. 또한, 시스템을 웹 기반 클라이언트-서버 모델로 구현하여 사용자가 언제 어디에서든 질의 필터링 시스템을 사용할 수 있게 하며, 웹 클라이언트를 통해 입력한 질의를 자동적 분석하는 쿼리 파서를 구현하여 별도의 프로그램의 수정 없이 질의 필터링을 적용할 수 있다.

Development of Big-data Management Platform Considering Docker Based Real Time Data Connecting and Processing Environments (도커 기반의 실시간 데이터 연계 및 처리 환경을 고려한 빅데이터 관리 플랫폼 개발)

  • Kim, Dong Gil;Park, Yong-Soon;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.4
    • /
    • pp.153-161
    • /
    • 2021
  • Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.

IoT data analytics architecture for smart healthcare using RFID and WSN

  • Ogur, Nur Banu;Al-Hubaishi, Mohammed;Ceken, Celal
    • ETRI Journal
    • /
    • v.44 no.1
    • /
    • pp.135-146
    • /
    • 2022
  • The importance of big data analytics has become apparent with the increasing volume of data on the Internet. The amount of data will increase even more with the widespread use of Internet of Things (IoT). One of the most important application areas of the IoT is healthcare. This study introduces new real-time data analytics architecture for an IoT-based smart healthcare system, which consists of a wireless sensor network and a radio-frequency identification technology in a vertical domain. The proposed platform also includes high-performance data analytics tools, such as Kafka, Spark, MongoDB, and NodeJS, in a horizontal domain. To investigate the performance of the system developed, a diagnosis of Wolff-Parkinson-White syndrome by logistic regression is discussed. The results show that the proposed IoT data analytics system can successfully process health data in real-time with an accuracy rate of 95% and it can handle large volumes of data. The developed system also communicates with a riverbed modeler using Transmission Control Protocol (TCP) to model any IoT-enabling technology. Therefore, the proposed architecture can be used as a time-saving experimental environment for any IoT-based system.

Trying to Place Beckett's View on Death in Western Thanatology (서구 죽음학에서 베케트 죽음관 자리매기기)

  • Hwang, Hoon-Sung
    • Journal of English Language & Literature
    • /
    • v.58 no.4
    • /
    • pp.611-632
    • /
    • 2012
  • Beckett's life-long struggling with death may be illuminated in terms of the Western tradition of thanatology as well as Philippe Ariès's anthropological classification of death. Among the Western tradition, Beckett's oeuvre incarnates memento mori, timor mori, nihilism, theatrum mundi, life as afterlife, and the transsubstantiation of the self. Among the five views of death Ariès suggests, Beckett appears to foreground the death of the self and the invisible dirty death. In a world devoid of transcendental Signified, Beckett's resident is "a poor player/That struts and frets his hour upon the stage." Our contemporary vision of death is dominated by the dirty death and timor mori resurrected from the cultural icon of danse macabre in the late Mediaeval age as vividly dramatized in W;t by Margaret Edson. Beckett stands in no man's land: Lucky complains of divine aphathia as well as scopes at the possibility of God's existence like Hamm. Beckett's way of getting out of the dilemma is laughing a mirthless and dianoetic laugher. To bourgeois class who shudder at the sight of Grim Death after forgettable years of indulgence and addiction to capitalist consumption, Beckett seems to preach, your life is a death-in-life, you are not born yet until you are baptized with existential awakening as Gregor Samsa in Kafka's Verwandlung, or Tolstoy in Confession.

A Study on the Real-time user purchase pattern analysis User Product Recommendation System in E-Commerce Environment (E-commerce 환경에서 실시간 사용자 구매 패턴 분석을 통한 사용자 상품 추천 시스템 연구)

  • Beom Jung Kim;Ji Hye Huh;Hyeopgeon Lee;Young Woon Kim
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2023.05a
    • /
    • pp.413-414
    • /
    • 2023
  • IT 기술의 발달로 E-Commerce 분야는 실시간으로 발생되는 데이터양이 증가하고 있으며, 발생된 데이터는 개인화 맞춤 서비스에 많이 활용되고 있다. 그러나 신생 E-commerce 기업은 신규 상품 및 기존 상품에 대한 정보와 고객 간의 상호 작용 데이터가 존재하지 않아 콜드 스타트 문제가 발생한다. 이에 본 논문에서는 E-commerce 환경에서 실시간 사용자 구매패턴 분석을 통한 사용자 상품 추천 시스템을 제안한다. 제안하는 시스템은 Kafka와 Spark를 사용해 실시간 스트림을 데이터를 처리한다. 주요 기능은 ALS 알고리즘과, FP-Growth 알고리즘을 적용해 콜트 스타트 문제를 해결하며, 사용자 구매 패턴 분석을 통한 분석 결과에 맞는 상품을 사용자에게 추천한다.

Design of a Disaster Big Data Platform for Collecting and Analyzing Social Media (소셜미디어 수집과 분석을 위한 재난 빅 데이터 플랫폼의 설계)

  • Nguyen, Van-Quyet;Nguyen, Sinh-Ngoc;Nguyen, Giang-Truong;Kim, Kyungbaek
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2017.04a
    • /
    • pp.661-664
    • /
    • 2017
  • Recently, during disasters occurrence, dealing with emergencies has been handled well by the early transmission of disaster relating notifications on social media networks (e.g., Twitter or Facebook). Intuitively, with their characteristics (e.g., real-time, mobility) and big communities whose users could be regarded as volunteers, social networks are proved to be a crucial role for disasters response. However, the amount of data transmitted during disasters is an obstacle for filtering informative messages; because the messages are diversity, large and very noise. This large volume of data could be seen as Social Big Data (SBD). In this paper, we proposed a big data platform for collecting and analyzing disasters' data from SBD. Firstly, we designed a collecting module; which could rapidly extract disasters' information from the Twitter; by big data frameworks supporting streaming data on distributed system; such as Kafka and Spark. Secondly, we developed an analyzing module which learned from SBD to distinguish the useful information from the irrelevant one. Finally, we also designed a real-time visualization on the web interface for displaying the results of analysis phase. To show the viability of our platform, we conducted experiments of the collecting and analyzing phases in 10 days for both real-time and historical tweets, which were about disasters happened in South Korea. The results prove that our big data platform could be applied to disaster information based systems, by providing a huge relevant data; which can be used for inferring affected regions and victims in disaster situations, from 21.000 collected tweets.