• Title/Summary/Keyword: Node distribution

Search Result 596, Processing Time 0.022 seconds

Implementation of Efficient Distributed Crawler through Stepwise Crawling Node Allocation

  • Kim, Hyuntae;Byun, Junhyung;Na, Yoseph;Jung, Yuchul
    • Journal of Advanced Information Technology and Convergence
    • /
    • v.10 no.2
    • /
    • pp.15-31
    • /
    • 2020
  • Various websites have been created due to the increased use of the Internet, and the number of documents distributed through these websites has increased proportionally. However, it is not easy to collect newly updated documents rapidly. Web crawling methods have been used to continuously collect and manage new documents, whereas existing crawling systems applying a single node demonstrate limited performances. Furthermore, crawlers applying distribution methods exhibit a problem related to effective node management for crawling. This study proposes an efficient distributed crawler through stepwise crawling node allocation, which identifies websites' properties and establishes crawling policies based on the properties identified to collect a large number of documents from multiple websites. The proposed crawler can calculate the number of documents included in a website, compare data collection time and the amount of data collected based on the number of nodes allocated to a specific website by repeatedly visiting the website, and automatically allocate the optimal number of nodes to each website for crawling. An experiment is conducted where the proposed and single-node methods are applied to 12 different websites; the experimental result indicates that the proposed crawler's data collection time decreased significantly compared with that of a single node crawler. This result is obtained because the proposed crawler applied data collection policies according to websites. Besides, it is confirmed that the work rate of the proposed model increased.

Optimal Distributed Computation with Communication Delays (통신지연이 있는 분산처리의 최적화)

  • Kim, Hyoung-Joong;Jee, Gyu-In;Lee, Jang-Gyu
    • Proceedings of the KIEE Conference
    • /
    • 1991.07a
    • /
    • pp.748-752
    • /
    • 1991
  • Tree network consisting of communicating processors is considered. The objective is to minimize the computation time by distributing the processing load to other nodes. The effect of the order of load distribution on the processing time is addressed. An algorithm which optimally determines the order of load distribution is developed. It is shown that the order depends only on the channel capacity between nodes but not on the computing capability of each node.

  • PDF

The magnetic energy distribution of asymmetrical & eccentrical the squirrel cage induction motor. (농형(籠型) 유도(誘導) 전동기(電動機)의 비대칭(非對稱), 편심구조(偏心構造)에 의한 자기(磁氣) 에너지 분포(分布))

  • Kim, Sang-Baek;Lee, Eun-Woong
    • Proceedings of the KIEE Conference
    • /
    • 1993.11a
    • /
    • pp.47-49
    • /
    • 1993
  • The squirrel cage three-phase induction motor has been used in all types of industry for a number of years. A series of noise components were identied analytically from the expressions for the node numbers and frequencies of the magnetic force waves, taking into account the stator and rotor sldtting, eccentricity, asymmetric and iron saturation. A method is presented for calculating the magnetic energy distribution of eccentrical and asymmetric the squirrel cage induction motor.

  • PDF

Moment Distribution in continuous Slabs of Simply Supported Bridges (단순 PC 빔교의 연속 바닥슬래브에 발생하는 모멘트 분포)

  • 최창근;송명관
    • Proceedings of the Korea Concrete Institute Conference
    • /
    • 1997.04a
    • /
    • pp.398-405
    • /
    • 1997
  • The finite element program is presented for the analysis of the moment distribution in continuous slabs of simply supported girders. The program includes the material nonlinearity of the continuous and steel reinforcements of the RC slabs, but assumes that the PC beam and cross beam behave linearly. In modeling the PC slabs and girders, the four node degenerated shell element formulated based on the assumed strain interpolation and the 3D beam element are used, respectively. Using the program, the influence of the existence of the cross bean, the filling of open joints on the continuous at supports, and perfect continuation of precast girder elements are examined.

  • PDF

Analysis of Hydroelastic Responses for Very Large Floating Structures with a Shallow Draft (천흘수 초대형 부유식 해양규조물의 유탄성 응답해석)

  • 신현경
    • Journal of Ocean Engineering and Technology
    • /
    • v.14 no.2
    • /
    • pp.53-59
    • /
    • 2000
  • A numerical method to predict responses of very large floating structures in wave is suggested using source-dipole distribution method. The deflection of the plate is calculated by the finite element method in terms of rigidity matrix of each node. The calculated results for a plate are compared with the experimental ones.

  • PDF

ANALYSIS OF LOOPED WATER DISTRIBUTION NETWORK

  • Ioan Sarbu
    • Water Engineering Research
    • /
    • v.2 no.3
    • /
    • pp.171-178
    • /
    • 2001
  • There are three methods for analyzing flow and pressure distribution in looped water distribution networks (the loop method, the node method, the element method) taking into consideration hydraulic parameters chosen as unknown. For all these methods the non-linear system of equations can be solved by iterative procedures. The paper presents a different approach to this problem by using the method of variational formulations for hydraulic analysis of water distribution networks. This method has the advantage that it uses a specialized optimization algorithm which minimizes directly an objective multivariable function without constraints, implemented in a computer program. The paper compares developed method to the classic Hardy-Cross method. This shows the good performance of the new method.

  • PDF

Task Distribution Scheme based on Service Requirements Considering Opportunistic Fog Computing Nodes in Fog Computing Environments (포그 컴퓨팅 환경에서 기회적 포그 컴퓨팅 노드들을 고려한 서비스 요구사항 기반 테스크 분배 방법)

  • Kyung, Yeunwoong
    • Journal of Korea Multimedia Society
    • /
    • v.24 no.1
    • /
    • pp.51-57
    • /
    • 2021
  • In this paper, we propose a task distribution scheme in fog computing environment considering opportunistic fog computing nodes. As latency is one of the important performance metric for IoT(Internet of Things) applications, there have been lots of researches on the fog computing system. However, since the load can be concentrated to the specific fog computing nodes due to the spatial and temporal IoT characteristics, the load distribution should be considered to prevent the performance degradation. Therefore, this paper proposes a task distribution scheme which considers the static as well as opportunistic fog computing nodes according to their mobility feature. Especially, based on the task requirements, the proposed scheme supports the delay sensitive task processing at the static fog node and delay in-sensitive tasks by means of the opportunistic fog nodes for the task distribution. Based on the performance evaluation, the proposed scheme shows low service response time compared to the conventional schemes.

Two Korean Species of Centroceras Kutzing (Ceramiaceae, Rhodophyta) (한국산 홍조 Centroceras속 이종)

  • 부성민
    • Journal of Plant Biology
    • /
    • v.28 no.4
    • /
    • pp.297-304
    • /
    • 1985
  • Two Korean species of Centroceras K tzing, Ceramiaceae was investigated taxonomically. C. clavulatum (Ag.) Montagne collected at several sited along the coast of Korea was characterized by regular dischotomous branches with whorl spines at every node, whereas, C. distichum Okamura collected at Soando in the southern coast was by alternate branches with gland cells around nodes. Biogeographic data show that Korea is almost northern limit in distribution of the former species.

  • PDF

A study on improving fairness and congestion control of DQDB using buffer threshold value (버퍼의 문턱치값을 이용한 DQDB망의 공평성 개선 및 혼잡 제어에 관한 연구)

  • 고성현;조진교
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.22 no.4
    • /
    • pp.618-636
    • /
    • 1997
  • DQDB(Distributed Queue Dual Bus) protocol, the IEEE 802.6 standard protocol for metropolitan area networks, does not fully take advantage of the capabilities of dual bus architecture. Although fairness in bandwidth distribution among nodes is improved when using so called the bandwidth balancing mechanism, the protocol requires a considerable amount of time to adjust to changes in the network load. Additionally, the bandwidth balancing mechanism leaves a portion of the available bandwidth unused. In a high-speed backbone network, each node may act as a bridge/ router which connects several LANs as well as hosts. However, Because the existence of high speed LANs becomes commonplace, the congestionmay occur on a node because of the limitation on access rate to the backbone network and on available buffer spaces. to release the congestion, it is desirable to install some congestion control algorithm in the node. In this paper, we propose an efficient congestion control mechanism and fair and waster-free MAC protocol for dual bus network. In this protocol, all the buffers in the network can be shared in such a way that the transmission rate of each node can be set proportional to its load. In other words, a heavily loaded node obtains a larger bandwidth to send the sements so tht the congestion can be avoided while the uncongested nodes slow down their transmission rate and store the incoming segments into thier buffers. this implies that the buffers on the network can be shared dynamically. Simulation results show that the proposed probotol significantly reduces the segment queueing delay of a heavily loaded node and segment loss rate when compared with original DQDB. And it enables an attractive high throughput in the backbone network. Because in the proposed protocol, each node does not send a requet by the segment but send a request one time in the meaning of having segments, the frequency of sending requests is very low in the proposed protocol. so the proposed protocol signigificantly reduces the segment queuing dely. and In the proposed protocol, each node uses bandwidth in proportion to its load. so In case of limitation of available buffer spaces, the proposed protocol reduces segment loss rate of a heavily loaded node. Bandwidth balancing DQDB requires the wastage of bandwidth to be fair bandwidth allocation. But the proposed DQDB MAC protocol enables fair bandwidth without wasting bandwidth by using bandwidth one after another among active nodes.

  • PDF

Dynamic Control of Random Constant Spreading Worm using Depth Distribution Characteristics

  • No, Byung-Gyu;Park, Doo-Soon;Hong, Min;Lee, Hwa-Min;Park, Yoon-Sok
    • Journal of Information Processing Systems
    • /
    • v.5 no.1
    • /
    • pp.33-40
    • /
    • 2009
  • Ever since the network-based malicious code commonly known as a 'worm' surfaced in the early part of the 1980's, its prevalence has grown more and more. The RCS (Random Constant Spreading) worm has become a dominant, malicious virus in recent computer networking circles. The worm retards the availability of an overall network by exhausting resources such as CPU capacity, network peripherals and transfer bandwidth, causing damage to an uninfected system as well as an infected system. The generation and spreading cycle of these worms progress rapidly. The existing studies to counter malicious code have studied the Microscopic Model for detecting worm generation based on some specific pattern or sign of attack, thus preventing its spread by countering the worm directly on detection. However, due to zero-day threat actualization, rapid spreading of the RCS worm and reduction of survival time, securing a security model to ensure the survivability of the network became an urgent problem that the existing solution-oriented security measures did not address. This paper analyzes the recently studied efficient dynamic network. Essentially, this paper suggests a model that dynamically controls the RCS worm using the characteristics of Power-Law and depth distribution of the delivery node, which is commonly seen in preferential growth networks. Moreover, we suggest a model that dynamically controls the spread of the worm using information about the depth distribution of delivery. We also verified via simulation that the load for each node was minimized at an optimal depth to effectively restrain the spread of the worm.