• Title/Summary/Keyword: NameNode

Search Result 39, Processing Time 0.025 seconds

A Study on the Effect of the Name Node and Data Node on the Big Data Processing Performance in a Hadoop Cluster (Hadoop 클러스터에서 네임 노드와 데이터 노드가 빅 데이터처리 성능에 미치는 영향에 관한 연구)

  • Lee, Younghun;Kim, Yongil
    • Smart Media Journal
    • /
    • v.6 no.3
    • /
    • pp.68-74
    • /
    • 2017
  • Big data processing processes various types of data such as files, images, and video to solve problems and provide insightful useful information. Currently, various platforms are used for big data processing, but many organizations and enterprises are using Hadoop for big data processing due to the simplicity, productivity, scalability, and fault tolerance of Hadoop. In addition, Hadoop can build clusters on various hardware platforms and handle big data by dividing into a name node (master) and a data node (slave). In this paper, we use a fully distributed mode used by actual institutions and companies as an operation mode. We have constructed a Hadoop cluster using a low-power and low-cost single board for smooth experiment. The performance analysis of Name node is compared through the same data processing using single board and laptop as name nodes. Analysis of influence by number of data nodes increases the number of data nodes by two times from the number of existing clusters. The effect of the above experiment was analyzed.

A Design of the Small File Grouping System Based on Naive Bayesian Classifier Model (나이브 베이지안 분류기 모델 기반의 소용량 파일 그룹화 시스템 설계)

  • Kim, Min-Jae;Kim, Kyung-Tae;Youn, Hee-Young
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2014.07a
    • /
    • pp.221-222
    • /
    • 2014
  • 빠른 웹의 성장으로 대용량 데이터를 효과적으로 처리할 수 있는 플랫폼 기술에 대한 관심이 높아지고 있다. 특히, HDFS는 이상적인 분산 파일 시스템으로 각광받고 있으며 대용량 파일의 처리를 목적으로 개발되었다. 하지만, 실제 파일들의 집합에서 소용량 파일이 차지하는 비중은 높은 편이다. 많은 수의 소용량 파일은 HDFS 성능 감소에 치명적인 원인이 된다. 많은 수의 소용량 파일들이 HDFS에 저장된다면 NameNode의 메모리 소비량이 증가하게 되며 많은 수의 소용량 파일은 많은 수의 DataNode와 NameNode를 요구하므로 상대적으로 처리시간이 많이 소모된다. 따라서 본 논문에서는 HDFS에서 소용량 파일의 저장과 액세스 효율성을 향상시키기 위하여 나이브 베이지안 분류기 알고리즘을 적용한 파일 그룹화 시스템을 설계하였다.

  • PDF

A Design of Hadoop Security Protocol using One Time Key based on Hash-chain (해시 체인 기반 일회용 키를 이용한 하둡 보안 프로토콜 설계)

  • Jeong, Eun-Hee;Lee, Byung-Kwan
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.10 no.4
    • /
    • pp.340-349
    • /
    • 2017
  • This paper is proposed Hadoop security protocol to protect a reply attack and impersonation attack. The proposed hadoop security protocol is consists of user authentication module, public key based data node authentication module, name node authentication module, and data node authentication module. The user authentication module is issued the temporary access ID from TGS after verifing user's identification on Authentication Server. The public key based data node authentication module generates secret key between name node and data node, and generates OTKL(One-Time Key List) using Hash-chain. The name node authentication module verifies user's identification using user's temporary access ID, and issues DT(Delegation Token) and BAT(Block Access Token) to user. The data node authentication module sends the encrypted data block to user after verifing user's identification using OwerID of BAT. Therefore the proposed hadoop security protocol dose not only prepare the exposure of data node's secret key by using OTKL, timestamp, owerID but also detect the reply attack and impersonation attack. Also, it enhances the data access of data node, and enforces data security by sending the encrypted data.

Domain name system for the efficient name service in mobile ad hoc networks (이동 애드혹 네트워크에서 효율적인 네임 서비스 제공을 위한 도메인 네임 시스템)

  • Ahn, Sang-Hyun;Lim, Yu-Jin;Kim, Sung-Rim
    • 전자공학회논문지 IE
    • /
    • v.44 no.1
    • /
    • pp.20-26
    • /
    • 2007
  • Most researches on the mobile ad hoc network (MANET) have been focused on routing protocols, but for the real service provision DNS(Domain Name System) has to be supported first. Due to the inherent characteristics of the mobile ad hoc network, the DNS of the wired network is assumed to be not good for the MANET environment. The approach of distributed DNSs can easily adapt to the node mobility, but incurs the name conflict resolution overhead. On the other hand, the centralized approach performs the name resolution based on the unicast communication without causing the name conflict resolution overhead. The most important issue of the centralized approach is to provide the seamless name resolution service under server mobility. Therefore, in this paper, we propose a new centralized DNS, Manet DNS, which works efficiently on name allocation and management and solves the network merging and partitioning problem as well as providing the seamless name resolution service.

Bacterial Node Soft Rot of Pepper (Capsicum annuum L.) Caused by Erwinia carotovora subsp. carotovora (Erwinia carotovora subsp. carotovora에 의한 고추 마디 무름병)

  • 정기채;임진우;김승한;임양숙;김종완
    • Korean Journal Plant Pathology
    • /
    • v.14 no.6
    • /
    • pp.741-743
    • /
    • 1998
  • A bacterial disease of pepper (Capsicum annuum L.) that rooted the stem nodes to black was found in pepper plants which cultivated in plastic house at Chungdo, Kyungpook, Korea in March, 1998. Bacterial isolates derived from the diseased peppers were pathogenic to potato, eggplant and Chinese cabbage but, was not pathogenic to chrysanthemum by artificial inoculation. On the basis of bacteriological characteristics and pathogenicity test on host plants, the causal organism of the node soft rot of pepper is identified as Erwinia carotovora subsp. carotovora and the name of disease is proposed as bacterial node soft rot of pepper.

  • PDF

Efficient routing in multicast mesh by using forwarding nodes and weighted cost function

  • Vyas, Kapila;Khuteta, Ajay;Chaturvedi, Amit
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.12
    • /
    • pp.5928-5947
    • /
    • 2019
  • Multicast Mesh based Mobile Ad-hoc NETworks (MANETs) provide efficient data transmission in energy restraint areas without a fixed infrastructure. In this paper, the authors present an improved version of protocol SLIMMER developed by them earlier, and name it SLIMMER-SN. Most mesh-based protocols suffer from redundancy; however, the proposed protocol controls redundancy through the concept of forwarding nodes. The proposed protocol uses remaining energy of a node to decide its energy efficiency. For measuring stability, a new metric called Stability of Node (SN) has been introduced which depends on transmission range, node density and node velocity. For data transfer, a weighted cost function selects the most energy efficient nodes / most stable nodes or a weighted combination of both. This makes the node selection criteria more dynamic. The protocol works in two steps: (1) calculating SN and (2) using SN value in the weighted cost function for selection of nodes. The study compared the proposed protocol, with other mesh-based protocols PUMA and SLIMMER, based on packet delivery ratio (PDR), throughput, end-to-end delay and average energy consumption under different simulation conditions. Results clearly demonstrate that SLIMMER-SN outperformed both PUMA and SLIMMER.

Data Dissemination in LTE-D2D Based Vehicular Network (LTE-D2D 차량 네트워크에서 정보 전달 방법)

  • Shim, Yong-Hui;Kim, Young-Han
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.40 no.3
    • /
    • pp.602-612
    • /
    • 2015
  • Current IEEE 802.11p which is suggested for vehicle to vehicle communication supports one hop communication. Thus, it has a limitation to carry out efficient data dissemination. In this thesis, we suggest LTE-D2D based vehicle network to provide efficient data dissemination in the vehicle environment. In this network architecture, we use name based message with IP packet options and we put the intermediate vehicle node called 'super vehicle node' and each normal vehicle node in the cell requests data to the super vehicle node. The super vehicle node responses data to the normal vehicle node. Performance analysis is based mathematical modeling. We compare LTE cellular network to LTE-D2D based vehicle network about throughput according to packet delivery time.

A Study on the Hierarchical Representation of Images: An Efficient Representation of Quadtrees BF Linear Quadtree (화상의 구조적 표현에 관한 연구- 4진트리의 효율적인 표현법:BF선형 4진트)

  • Kim, Min-Hwan;Han, Sang-Ho;Hwang, Hee-Yeung
    • The Transactions of the Korean Institute of Electrical Engineers
    • /
    • v.37 no.7
    • /
    • pp.498-509
    • /
    • 1988
  • A BF(breadth-first) linear quadtree as a new data structure for image data is suggested, which enables us to compress the image data efficiently and to make operations of the compressed data easily. It is a list of path names for black nodes as the linear quadtree is. The path name for each black node of a BF linear quadtree is represented as a sequence of path codes from the root node to itself, whereas that of linear quadtree as a sequence of path codes from the root node to itself and fill characters for cut-offed path from it to any n-level node which corresponds to a pixel of an image. The BF linear quadtree provides a more efficent compression ratio than the linear quadtree does, because the former does not require redundant characters, fill characters, for the cut-offed paths. Several operations for image processing can be also implemented on this hierarchical structure efficiently, because it is composed of only the black nodes ad the linear quadtree is . In this paper, algorithms for several operations on the BF linear quadtree are defined and analyzed. Experimental results for forur image data are also given and discussed.

Collaboration and Node Migration Method of Multi-Agent Using Metadata of Naming-Agent (네이밍 에이전트의 메타데이터를 이용한 멀티 에이전트의 협력 및 노드 이주 기법)

  • Kim, Kwang-Jong;Lee, Yon-Sik
    • The KIPS Transactions:PartD
    • /
    • v.11D no.1
    • /
    • pp.105-114
    • /
    • 2004
  • In this paper, we propose a collaboration method of diverse agents each others in multi-agent model and describe a node migration algorithm of Mobile-Agent (MA) using by the metadata of Naming-Agent (NA). Collaboration work of multi-agent assures stability of agent system and provides reliability of information retrieval on the distributed environment. NA, an important part of multi-agent, identifies each agents and series the unique name of each agents, and each agent references the specified object using by its name. Also, NA integrates and manages naming service by agents classification such as Client-Push-Agent (CPA), Server-Push-Agent (SPA), and System-Monitoring-Agent (SMA) based on its characteristic. And, NA provides the location list of mobile nodes to specified MA. Therefore, when MA does move through the nodes, it is needed to improve the efficiency of node migration by specified priority according to hit_count, hit_ratio, node processing and network traffic time. Therefore, in this paper, for the integrated naming service, we design Naming Agent and show the structure of metadata which constructed with fields such as hit_count, hit_ratio, total_count of documents, and so on. And, this paper presents the flow of creation and updating of metadata and the method of node migration with hit_count through the collaboration of multi-agent.

XML Clustering Technique by Genetic Algorithm (유전자 알고리즘을 통한 XML 군집화 방법)

  • Kim, Woo-Saeng
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.49 no.3
    • /
    • pp.1-7
    • /
    • 2012
  • Recently, researches are studied in developing efficient techniques for accessing, querying, and managing XML documents which are frequently used in the Internet. In this paper, we propose a new method to cluster XML documents efficiently. An element of a XML document corresponds to a node of the corresponding tree and an inclusion relationship of the document corresponds to a relationship between parent and child node of the tree. Therefore, similar XML documents are similar to the node's name and level of the corresponding trees. We make evaluation function with this characteristic to cluster XML documents by genetic algorithm. The experiment shows that our proposed method has better performance than other existing methods.