• Title/Summary/Keyword: Hive

Search Result 81, Processing Time 0.028 seconds

Security Threats and Review for SQL on Hadoop (SQL on Hadoop 기술 동향 및 보안 위협)

  • Youn, Han Jung;Suk, Sang Kee
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.04a
    • /
    • pp.691-693
    • /
    • 2015
  • SQL on Hadoop 기술은 하둡 분산 파일 시스템에 저장된 데이터를 대상으로 SQL을 이용하여 사용자의 질의를 처리하는 기술이다. 기존의 Hadoop 시스템이 맵리듀스의 한계와 기존 시스템의 호환성으로 인해 RDBMS와 병행사용이 불가피하다는 단점을 SQL을 이용해 극복하고자 하는 것이다. 본 논문에서는 SQL on Hadoop의 대표적 프레임워크인 Hive와 Impala의 특징과, 연구동향에 대해 살펴보고 예상되는 보안 위협에 대해 고찰한다.

Visualization of Anomaly Detection in Hadoop System Information (하둡 시스템 정보의 이상탐지를 위한 시각화)

  • Yang, Seokwoo;Son, Siwoon;Gil, Myeong-Seon;Moon, Yang-Sae;Won, Hee-Sun
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.04a
    • /
    • pp.702-705
    • /
    • 2015
  • 본 논문에서는 하듐 환경에서 시스템 정보의 이상탐지를 위한 시각화 기능을 설계 및 구현한다. 제안한 이상탐지 시각화 기능은 크게 세 단계로 구분된다. 먼저, 각 노드로부터 시스템 로그 데이터(캐시 및 메인 메모리)를 수집하여 하이브(Hive) 저장한다. 그리고 저장한 데이터에 3-시그마 규칙을 적용하여 이상탐지를 수행한 후 관계형 데이터베이스에 적합하도록 재가공한다. 마지막으로, 스쿱(Sqoop)을 통해 RDBMS(MariaDB)에 이상탕지 결과를 저장하고, DHTMLX 차트 라이브러리를 사용하여 이를 시각화한다. 시각화 결과, 로그 데이터의 이상탐지와 데이터간의 상관관계를 직관적으로 이해할 수 있게 되었다.

Big data-based piping material analysis framework in offshore structure for contract design

  • Oh, Min-Jae;Roh, Myung-Il;Park, Sung-Woo;Chun, Do-Hyun;Myung, Sehyun
    • Ocean Systems Engineering
    • /
    • v.9 no.1
    • /
    • pp.79-95
    • /
    • 2019
  • The material analysis of an offshore structure is generally conducted in the contract design phase for the price quotation of a new offshore project. This analysis is conducted manually by an engineer, which is time-consuming and can lead to inaccurate results, because the data size from previous projects is too large, and there are so many materials to consider. In this study, the piping materials in an offshore structure are analyzed for contract design using a big data framework. The big data technologies used include HDFS (Hadoop Distributed File System) for data saving, Hive and HBase for the database to handle the saved data, Spark and Kylin for data processing, and Zeppelin for user interface and visualization. The analyzed results show that the proposed big data framework can reduce the efforts put toward contract design in the estimation of the piping material cost.

A study on the Beehive Door Opening and Closing System using a Hornet Sound Analysis

  • Kim, Joon Ho;Han, Wook;Chung, Wonki
    • International Journal of Advanced Culture Technology
    • /
    • v.10 no.3
    • /
    • pp.393-396
    • /
    • 2022
  • Recently, rapid climate change has had a significant impact on the ecosystem of honeybees. In addition, the problem of Vespa Hornets invasion of colonies has a fatal impact on the bee ecosystem, independent of climate change. Especially in late summer. This study relates to a method for preventing Vespa Hornets attack. In this study, we developed a Vespa Hornets sound detection device was developed by collecting and analyzing the sound of a Vespa Hornets and applying IoT technology. The developed device detects the sound of a Vespa Hornets when Vespa Hornets appears around the hive of the bees and sends a signal to automatically close the door of the beehive. The device that receives the signal drives the motor that controls the honeycomb door to close the beehive door. The Vespa Hornets sound detection device operates until no Vespa Hornets sound is detected. The system developed by us is expected to be installed in the beehives of actual beekeeping farms to dramatically reduce the damage caused by by Vespa Hornets.

Safety Autonomous Platform Design with Ensemble AI Models (앙상블 인공지능 모델을 활용한 안전 관리 자율운영 플랫폼 설계)

  • Dongyeop Lee;Daesik Lim;Soojeong Woo;Youngho Moon;Minjeong Kim;Joonwon Lee
    • Journal of Advanced Navigation Technology
    • /
    • v.28 no.1
    • /
    • pp.159-162
    • /
    • 2024
  • This paper proposes a novel safety autonomous platform (SAP) architecture that can automatically and precisely manage on-site safety through ensemble artificial intelligence models generated from video information, worker's biometric information, and the safety rule to estimate the risk index. We practically designed the proposed SAP architecture by the Hadoop ecosystem with Kafka/NiFi, Spark/Hive, Hue, ELK (Elasticsearch, Logstash, Kibana), Ansible, etc., and confirmed that it worked well with safety mobility gateways for providing various safety applications.

Comparison of the pollinating activities according to number of combs per honeybee (Apis mellifera) hive released in the strawberry (Seolhyang var.) vinyl-houses (서양종꿀벌(Apis mellifera L.) 소비매수별 시설딸기(설향) 화분매개활동 비교)

  • Lee, Sang-Beom;Yoon, Hyung-Joo;Kang, Seok Woo;Kwon, Jae Seok;Park, Ki-Kwan
    • Journal of Sericultural and Entomological Science
    • /
    • v.52 no.2
    • /
    • pp.134-141
    • /
    • 2014
  • This study was examined the pollinating activity and the economical effect according to numbers released of Apis mellifera in the $825m^2$ strawberry (Seolhyang var.) vinyl-houses. The time-zone of pollinating activity relative to numbers of honeycomb released at the strawberry (Seolhyang var.) vinyl-houses was together from 9A.M. to 4P.M., and the peak time of pollinating activity was 11A.M.. The effects on pollinating activity relative to the honeycomb numbers in the honeybee hive released at the strawberry houses were ordered 5 honeycombs (11,000 heads), 4 honeycombs (8,800 heads) and 3 honeycombs (6,600 heads). The rate of workers lost in A. mellifera hives with 5 honeybee combs and 4 honeycombs during the strawberry cultivating period were lower than that of 3 honeycombs. The rates of fruit set by pollinating activity relative to the honeycomb numbers in the honeybee hive released at the strawberry vinyl-houses were same level with over 98%. The fruit qualities; No. of seeds, sugar content and rate of normal fruit set were same level, but fruit weights were ordered 5 honeycombs in 37.2 g, 4 honeycombs in 35.6 g and 3 honeycombs in 32.6 g. The marketing incomes of 4 honeycombs and 5 honeycombs were 9% to 13% higher than that of 3 honeycombs, respectively. Therefore, when the strawberry (Seolhyang var.) was planted at $825m^2$ of a vinyl-houses, it was surveyed that the most suitable numbers of honeycomb were over 4 honeycombs (8,800 heads).

Foraging behaviour and preference of pollen sources by honey bee (Apis mellifera) relative to protein contents

  • Ghosh, Sampat;Jeon, Hyejin;Jung, Chuleui
    • Journal of Ecology and Environment
    • /
    • v.44 no.1
    • /
    • pp.26-32
    • /
    • 2020
  • Background: Pollen is an important source of protein and lipids for many animals including honey bees. In order to understand the foraging behaviour of honey bee colonies and preference among the available floral resources, pollen collections from three experimental healthy colonies of honey bees were analysed in the month of June. Results: The amount of pollen collections were related to the colony's need which was indicated by the number of larval and adult bees present in the hive. Interesting was the sequence of pollen collection from different floral sources. All honey bee colonies collected pollens from Trifolium repens first, then Erigeron annus and the third choice was Coreopsis drummondii and Oenothera biennis flowers. Total protein content of Trifolium pollen was the highest (20.0 g/100 g DM), and the others were in the range of 8.9-11.4 g/100 g DM. Conclusion: The results indicated that the first criteria for honey bee foraging preference of pollens would be the nutritional contents of protein and the resource availability of the lesser nutritious floral sources. This information can help pollinator protection programmes of habitat manipulation using flowering plants for nectar and pollen sources.

Hadoop-based Large Data Management and Analysis for Parking Enforcement System (주정차 단속 시스템을 위한 하둡 기반 대용량 데이터 관리 및 분석)

  • Baek, Na-Eun;Song, Youngho;Shin, Jaehwan;Chang, Jae-Woo
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2017.04a
    • /
    • pp.429-432
    • /
    • 2017
  • 자동차 보급률 증가로 인해 교통 혼잡, 불법 주정차 등의 사회적 문제가 발생하고 있다. 특히 불법 주정차는 교통 혼잡, 주차 공간 부족 등 부가적인 문제를 발생시키고 있다. 따라서 각 지방자치단체에서는 불법 주정차 문제를 해결하기 위한 방안을 연구하고 있다. 그러나 이러한 방안은 초기 비용 발생 및 인력 부족 등의 한계가 있다. 한편, 정보통신의 발달에 따라 공공 업무에도 대량의 공공데이터를 효율적으로 처리하기 위한 연구가 진행되고 있다. 하지만 이러한 연구 또한 빅데이터 처리 플랫폼 부족 및 분석 시스템이 미흡한 한계가 존재한다. 따라서 본 논문에서는 불법 주정차 데이터와 같은 공공 데이터를 효율적으로 처리하기 위해, 주정차 단속 시스템을 위한 하둡 기반 대용량 데이터 관리 및 분석 시스템을 제안한다. 제안하는 시스템은 첫째, 주차단속을 수행할 때 주차단속 데이터를 하이브(Hive)를 통해 저장하고, 단속된 차량의 차주를 검색하여 단속임을 알리거나 과태료를 부과한다. 둘째, 웹 인터페이스를 통해 수집된 주차단속 데이터에 대한 다양한 분석을 수행하고, 분석된 데이터에 대한 R을 이용한 시각화를 제공한다.

Users' Design Process in Immersive Environments (몰입형 환경에서의 사용자 디자인 과정에 관한 연구)

  • Cho, Myung Eun;Kim, MI Jeong
    • Korean Institute of Interior Design Journal
    • /
    • v.26 no.2
    • /
    • pp.64-71
    • /
    • 2017
  • The purpose of this study is to investigate the potential of immersive environments in the design domain by analyzing students' design process in immersive environments from cognitive and experiential aspects. To do this, we reviewed major concepts and theories such as users' immersion, participation, presence, and awareness and developed a comprehensive conceptual framework of immersive shared environments. In three different visualization systems of HIVE, teams consisting two students were assigned to design tasks and the design process was analyzed by a customized framework. The characteristics of the immersive environment related to the performance of the design task were different from those of the previous studies. The perception of the relationship between the spaces is very important, and the perception of the surrounding objects is interested in the shape or the material such as whether the object is flat or inclined. Also, it is found that the multi-technology of immersive environments is very useful for creative collaboration. In the future, a more comprehensive analysis of the effects on design decisions in a more diverse visual interface condition and the effects on more diverse design areas should be added.

Design and Implementation of a Content Manager in the Multimedia Steaming Framework (멀티미디어 스트리밍 프레임워크에서 컨텐츠 관리자의 설계 및 구현)

  • Hong, Yeong-Rae;Kim, Hyeong-Il;Lee, Seung-Ryong;Jeong, Byeong-Su;Yun, Seok-Hwan;Jeong, Chan-Geun
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.2S
    • /
    • pp.733-743
    • /
    • 2000
  • This paper describes design and implementation of a content manager in the Integrated Steaming Framework Architecture (ISSA) that is proposed by the authors. The ISSA can provide an environment to develop multimedia streaming applications under heterogeneous distributed systems. The goal of ISSA is to extend the limitations of existing streaming systems. It can support diverse media formats and high level programming environment for streaming application developers. Moreover, it is independent from underlying networks and operating systems, and compatible with the global real-time multimedia database system(BeeHive) so that streaming media is efficiently retrieved, stored, and serviced. The role of a content manager is important in the ISSA environment since it manages an information of media that are provided by the server, and allow users to access media more easily by means of conveying that information to the streaming server, Web server, and client efficiently. The proposed content manager is not only to meed these requirements, but also to provide streaming information to media source and transport manager in order to be an efficient streaming. Furthermore, it supports database transaction processing by using the database connector.

  • PDF