• Title/Summary/Keyword: Allocation loading

Search Result 66, Processing Time 0.024 seconds

Design and Implementation of a High Performance Web Crawler (고성능 웹크롤러의 설계 및 구현)

  • Kim Hie-Cheol;Chae Soo-Hoan
    • Journal of Digital Contents Society
    • /
    • v.4 no.2
    • /
    • pp.127-137
    • /
    • 2003
  • A Web crawler is an important Internet software technology used in a variety of Internet application software which includes search engines. As Internet continues to grow, implementations of high performance web crawlers are urgently demanded. In this paper, we study how to support dynamic scheduling for a multiprocess-based web crawler. For high performance, web crawlers are usually based on multiprocess in their implementations. In these systems, crawl scheduling which manages the allocation of web pages to each process for loading is one of the important issues. In this paper, we identify issues which are important and challenging in the crawl scheduling. To address the issue, we propose a dynamic crawl scheduling framework and subsequently a system architecture for a web crawler with dynamic crawl scheduling support. This paper presents the design of the Web crawler with dynamic scheduling support.

  • PDF

Design and Implementation of a High Performance Web Crawler (고성능 웹크롤러의 설계 및 구현)

  • 권성호;이영탁;김영준;이용두
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.8 no.4
    • /
    • pp.64-72
    • /
    • 2003
  • A Web crawler is an important Internet software technology used in a variety of Internet application software which includes search engines. As Internet continues to grow, implementations of high performance web crawlers are urgently demanded. In this paper, we study how to support dynamic scheduling for a multiprocess-based web crawler. For high peformance, web crawlers are usually based on multiprocess in their implementations. In these systems, crawl scheduling which manages the allocation of web pages to each process for loading is one of the important issues. In this paper, we identify issues which are important and challenging in the crawl scheduling. To address the issue, we propose a dynamic crawl scheduling framework and subsequently a system architecture for a web crawler with dynamic crawl scheduling support. And we analysed the behaviors of Web crawler. Based on the analysis result, we suggest the direction for the design of high performance Web crawler.

  • PDF

Distributed Processing of Big Data Analysis based on R using SparkR (SparkR을 이용한 R 기반 빅데이터 분석의 분산 처리)

  • Ryu, Woo-Seok
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.17 no.1
    • /
    • pp.161-166
    • /
    • 2022
  • In this paper, we analyze the problems that occur when performing the big data analysis using R as a data analysis tool, and present the usefulness of the data analysis with SparkR which connects R and Spark to support distributed processing of big data effectively. First, we study the memory allocation problem of R which occurs when loading large amounts of data and performing operations, and the characteristics and programming environment of SparkR. And then, we perform the comparison analysis of the execution performance when linear regression analysis is performed in each environment. As a result of the analysis, it was shown that R can be used for data analysis through SparkR without additional language learning, and the code written in R can be effectively processed distributedly according to the increase in the number of nodes in the cluster.

Spectral Efficiency of MC-CDMA (MC-CDMA 방식의 주파수 효율)

  • Han Hee-Goo;Oh Seong-Keun
    • Journal of the Institute of Electronics Engineers of Korea TC
    • /
    • v.43 no.3 s.345
    • /
    • pp.39-48
    • /
    • 2006
  • In this paper, we analyze the spectral efficiency of multicarrier-code division multiple access (MC-CDMA) scheme. First, we derive a generalized formula for the spectral efficiency according to the number of subcarriers involved in, code division multiplexing and the number of codes used (i.e., loading factor), under a given set of channel coefficients. Also, we derive a generalized formula for spectral efficiency of various reduced-complexity systems that divide the full sets of subcarriers into several groups of subcarriers for code division multiplexing. Then, through these derivations, we establish an inter-relationship between the frequency selectivity and diversity order according to the number of multipaths. From the results, we choose the smallest code length while maximizing the diversity effect, provide an optimum subcarrier allocation strategy, and finally suggest a system structure for capacity-maximizing under the smallest code length. Through numerical analyses under simulated environments, we analyze the properties of spectral efficiency of various systems with reduced complexity and choose a major contributing factors to system design and a better system design methodology. Finally, we compare the spectral efficiency of the MC-CDMA scheme and orthogonal frequency division multiplexing (OFDM) scheme to make a relationship between both schemes.

A Study on the Judgement Criterion of Arrived Ship under Voyage Charterparty (항해용선계약상 도착선의 판단기준에 관한 연구)

  • Han, Nakhyun;Lee, Jaesung
    • Journal of Korea Port Economic Association
    • /
    • v.28 no.3
    • /
    • pp.167-192
    • /
    • 2012
  • The purpose of the study aims to analyse the judgement criterion of arrived ship under voyage charterparty with the Merida Case. A ship is an arrived ship if she is in port and either able to proceed immediately to a berth or in such a position that she is at the immediate and effective disposition of the chaterparty. Identification of the specified destination-whether berth or port-impacts on the incidence of loss occasioned by delay in loading or discharging, when the delay is due to the place at which the vessel is obliged by the terms of the charterparty to load or discharge her cargo being occupied by other shipping. The Merida case is an appeal by the charterers from a final Arbitration award of two very experienced arbitrators, dated 20th April, 2009. The arbitrators held that a voyage charterparty, dated 5th February, 2007, of the vessel, The M/V Merida, entered into between charterers and the owners, was a port rather than a berth Charterparty. The Primary relevance of this distinction does to the allocation, as between owners and charterers, of the risk of delay caused by congestion at load and discharge ports. The question of law arising in this appeal is whether the arbitrators were right to conclude that the charterparty was a port and not a berth charterparty. The arbitrators additionary placed some reliance on a post-contractual e-mail from the agents, which suggested that charterers did not dispute the validity of the NOR-and, hence, that this was a port charterparty.

Simulation-based Production Analysis of Food Processing Plant Considering Scenario Expansion (시나리오 확장을 고려한 식품 가공공장의 시뮬레이션 기반 생산량 분석)

  • Yeong-Hyun Lim ;Hak-Jong, Joo ;Tae-Kyung Kim ;Kyung-Min Seo
    • Journal of the Korea Society for Simulation
    • /
    • v.32 no.3
    • /
    • pp.93-108
    • /
    • 2023
  • In manufacturing productivity analysis, understanding the intricate interplay among factors like facility performance, layout design, and workforce allocation within the production line is imperative. This paper introduces a simulation-based methodology tailored to food manufacturing, progressively expanding scenarios to analyze production enhancement. The target system is a food processing plant, encompassing production processes, including warehousing, processing, subdivision, packaging, inspection, loading, and storage. First, we analyze the target system and design a simulation model according to the actual layout arrangement of equipment and workers. Then, we validate the developed model reflecting the real data obtained from the target system, such as the workers' working time and the equipment's processing time. The proposed model aims to identify optimal factor values for productivity gains through incremental scenario comparisons. To this end, three stages of simulation experiments were conducted by extending the equipment and worker models of the subdivision and packaging processes. The simulation experiments have shown that productivity depends on the placement of skilled workers and the performance of the packaging machine. The proposed method in this study will offer combinations of factors for the specific production requirements and support optimal decision-making in the real-world field.