• Title/Summary/Keyword: Crawler

Search Result 199, Processing Time 0.025 seconds

KONG-DB: Korean Novel Geo-name DB & Search and Visualization System Using Dictionary from the Web (KONG-DB: 웹 상의 어휘 사전을 활용한 한국 소설 지명 DB, 검색 및 시각화 시스템)

  • Park, Sung Hee
    • Journal of the Korean Society for information Management
    • /
    • v.33 no.3
    • /
    • pp.321-343
    • /
    • 2016
  • This study aimed to design a semi-automatic web-based pilot system 1) to build a Korean novel geo-name, 2) to update the database using automatic geo-name extraction for a scalable database, and 3) to retrieve/visualize the usage of an old geo-name on the map. In particular, the problem of extracting novel geo-names, which are currently obsolete, is difficult to solve because obtaining a corpus used for training dataset is burden. To build a corpus for training data, an admin tool, HTML crawler and parser in Python, crawled geo-names and usages from a vocabulary dictionary for Korean New Novel enough to train a named entity tagger for extracting even novel geo-names not shown up in a training corpus. By means of a training corpus and an automatic extraction tool, the geo-name database was made scalable. In addition, the system can visualize the geo-name on the map. The work of study also designed, implemented the prototype and empirically verified the validity of the pilot system. Lastly, items to be improved have also been addressed.

Effective Web Crawling Orderings from Graph Search Techniques (그래프 탐색 기법을 이용한 효율적인 웹 크롤링 방법들)

  • Kim, Jin-Il;Kwon, Yoo-Jin;Kim, Jin-Wook;Kim, Sung-Ryul;Park, Kun-Soo
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.37 no.1
    • /
    • pp.27-34
    • /
    • 2010
  • Web crawlers are fundamental programs which iteratively download web pages by following links of web pages starting from a small set of initial URLs. Previously several web crawling orderings have been proposed to crawl popular web pages in preference to other pages, but some graph search techniques whose characteristics and efficient implementations had been studied in graph theory community have not been applied yet for web crawling orderings. In this paper we consider various graph search techniques including lexicographic breadth-first search, lexicographic depth-first search and maximum cardinality search as well as well-known breadth-first search and depth-first search, and then choose effective web crawling orderings which have linear time complexity and crawl popular pages early. Especially, for maximum cardinality search and lexicographic breadth-first search whose implementations are non-trivial, we propose linear-time web crawling orderings by applying the partition refinement method. Experimental results show that maximum cardinality search has desirable properties in both time complexity and the quality of crawled pages.

On the vibration influence to the running power plant facilities when the foundation excavated of the cautious blasting works (삼천포화력발전소 3, 4호기 증설에 따르는 정밀발파작업으로 인한 인접가동발전기 및 구조물에 미치는 진동영향조사)

  • Huh, Ginn
    • Journal of the Korean Professional Engineers Association
    • /
    • v.24 no.6
    • /
    • pp.97-105
    • /
    • 1991
  • The cautious blasting works had been used with emulsion explosion electric M/S delay caps. Drill depth was from 3m to 6m with Crawler Drill ø70mm on the calcalious sand stone (soft-moderate-semi hard Rock). The total numbers of fire blast were 88 round. Scale distance were induces 15.52-60.32. It was applied to propagation Law in blasting vibration as follows. Propagation Law in Blasting Vibration (Equation omitted) where V : Peak partical velocity(cm/sec) D : Distance between explosion and recording sites(m) W : Maximum Charge per delay-period of eighit milliseconds o. more(kg) K : Ground transmission constant, empirically determind on the Rocks, Explosive and drilling pattern ets. b : Charge exponents n : Reduced exponents Where the quantity D / W$^n$ is known as the Scale distance. Above equation is worked by the U.S Bureau of Mines to determine peak particle velocity. The propagation Law can be catagrorized in three graups. Cubic root Scaling charge per delay Square root Scaling of charge per delay Site-specific Scaling of charge per delay Charge and reduction exponents carried out by multiple regressional analysis. It's divided into under loom and over 100m distance because the frequency is verified by the distance from blast site. Empirical equation of cautious blasting vibration is as follows. Over 30 ‥‥‥under 100m ‥‥‥V=41(D/$^3$√W)$\^$-1.41/ ‥‥‥A Over 100 ‥‥‥‥under 100m ‥‥‥V=121(D/$^3$√W)$\^$-1.56/ ‥‥‥B K value on the above equation has to be more specified for furthur understang about the effect of explosives, Rock strength. And Drilling pattern on the vibration levels, it is necessary to carry out more tests.

  • PDF

On the vibration influence to the running power plant facilities when the foundation excavated of the cautious blasting works. (S 화력발전소 3, 4호기 증설에 따르는 정밀발파작업으로 인한 인접가동발전기 및 구조물에 미치는 진동영향조사)

  • Huh Ginn
    • Explosives and Blasting
    • /
    • v.9 no.4
    • /
    • pp.3-12
    • /
    • 1991
  • The cautious blasting works had been used with emulsion explosion electric M /S delay caps. Drill depth was from 3m to 6m with Crawler Drill 70mm on the calcalious sand stone (soft-moderate-semi hard Rock) . The total numbers of feet blast were 88. Scale distance were induces 15.52-60.32. It was applied to Propagation Law in blasting vibration as follows .Propagtion Law in Blasting Vibration V=k(D/W/sup b/)/sup n/ where V : Peak partical velocity(cm/sec) D : Distance between explosion and recording sites(m) W ; Maximum Charge per delay -period of eight milliseconds or more(Kg) K : Ground transmission constant, empirically determind on the Rocks, Explosive and drilling pattern ets. b : Charge exponents n : Reduced exponents Where the quantity D/W/sup b/ is known as the Scale distance. Above equation is worked by the U.S Bureau of Mines to determine peak particle velocity. The propagation Law can be catagrorized in three groups. Cabic root Scaling charge per delay Square root Scaling of charge per delay Site-specific Scaling of charge delay Charge and reduction exponents carried out by multiple regressional analysis. It's divided into under loom and over loom distance because the frequency is varified by the distance from blast site. Empirical equation of cautious blasting vibration is as follows. Over 30m--under 100m----V=41(D/ W)/sup -1.41/-----A Over l00m---------V=121(D/ W)/sup -1.56/-----B K value on the above equation has to be more specified for furthur understand about the effect of explosives. Rock strength, And Drilling pattern on the vibration levels, it is necessary to carry out more tests.

  • PDF

A Job Allocation Manager for Dynamic Remote Execution of Distributed Jobs in P2P Network (분산처리 작업의 동적 원격실행을 위한 P2P 기반 작업 할당 관리자)

  • Lee, Seung-Ha;Kim, Yang-Woo
    • Journal of Internet Computing and Services
    • /
    • v.7 no.6
    • /
    • pp.87-103
    • /
    • 2006
  • Advances in computer and network technology provide new computing environment that were only possible with supercomputers before. In order to provide the environment, a distributed runtime system has to be provided, but most of the conventional distributed runtime systems lack in providing dynamic and flexible system reconfiguration depending on workload variance, due to a static architecture of fixed master node and slave working nodes. This paper proposes and implements a new model for distributed job allocation and management which is a distributed runtime system is P2P environment for flexible and dynamic system reconfiguration. The implemented systems enables job program transfer and management, remote compile and execution among cooperative developers based on P2P standard protocol Jxta platform. Since it makes dynamic and flexible system reconfiguration possible, the proposed method has some advantages in that it can collect and utilize idle computing resources immediately at a needed time for distributed job processing. Moreover, the implemented system's effectiveness and performance increase are shown by applying and processing the crawler jobs, in a distributed way, for collecting a large amount of data needed for internet search.

  • PDF

A Crowdsourcing-based Emotional Words Tagging Game for Building a Polarity Lexicon in Korean (한국어 극성 사전 구축을 위한 크라우드소싱 기반 감성 단어 극성 태깅 게임)

  • Kim, Jun-Gi;Kang, Shin-Jin;Bae, Byung-Chull
    • Journal of Korea Game Society
    • /
    • v.17 no.2
    • /
    • pp.135-144
    • /
    • 2017
  • Sentiment analysis refers to a way of analyzing the writer's subjective opinions or feelings through text. For effective sentiment analysis, it is essential to build emotional word polarity lexicon. This paper introduces a crowdsourcing-based game that we have developed for efficiently building a polarity lexicon in Korean. First, we collected a corpus from the relating Internet communities using a crawler, and we classified them into words using the Twitter POS analyzer. These POS-tagged words are provided as a form of mobile platform based tagging game in which the players voluntarily tagged the polarities of the words, and then the result was collected into the database. So far we have tagged the polarities of about 1200 words. We expect that our research can contribute to the Korean sentiment analysis research especially in the game domain by collecting more emotional word data in the future.

Development and Reproductive Capacity of Protopulvinaria mangiferae (Green) (Homoptera: Coccidae) (담팔수깍지벌레의 발육과 증식능력)

  • 김종국
    • Korean journal of applied entomology
    • /
    • v.36 no.1
    • /
    • pp.43-47
    • /
    • 1997
  • This study was carried out in the laboratory to clarify effects of different temperature of Protopulvinaria mangiferae(Green) on development, survivorship and reproduction. Developmental period of the mango shieldscale from crawlers to preoviposition adult decreased as temperature increased. The threshold temperature andthermal constants for the development of one generation were 11.7"C and 1000.0 day-degrees, respectively. At25$^{\circ}$C and 30$^{\circ}$C, survival rate from egg to preoviposition adult were 82% and 6096, respectively. Hatchability ofeggs was more than 99% at both condition. The reproductive period overaged 50 days(2SnC) and 33 days(30$^{\circ}$C). After mature adult began to reporduce, and more than 50% of the crawlers emerged during the firsthalf of their life time. The net reproduction rate per generation(R), mean length of a generation0 and intrinsicrate of natural increase(r, ) were higher at 25$^{\circ}$C than at 30"C, and the values measured at 25$^{\circ}$C were 132.6, 76.2 and 0.064/female/day, respectively.

  • PDF

A collaborative simulation in shipbuilding and the offshore installation based on the integration of the dynamic analysis, virtual reality, and control devices

  • Li, Xing;Roh, Myung-Il;Ham, Seung-Ho
    • International Journal of Naval Architecture and Ocean Engineering
    • /
    • v.11 no.2
    • /
    • pp.699-722
    • /
    • 2019
  • It is difficult to observe the potential risks of lifting or turn-over operations in the early stages before a real operation. Therefore, many dynamic simulations have been designed to predict the risks and to reduce the possibility of accidents. These simulations, however, have usually been performed for predetermined and fixed scenarios, so they do not reflect the real-time control of an operator that is one of the most important influential factors in an operation; additionally, lifting or turn-over operations should be a collaboration involving more than two operators. Therefore, this study presents an integrated method for a collaborative simulation that allows multiple workers to operate together in the virtual world. The proposed method is composed of four components. The first component is a dynamic analysis that is based on multibody-system dynamics. The second component is VR (virtual reality) for the generation of realistic views for the operators. The third component comprises the control devices and the scenario generator to handle the crane in the virtual environment. Lastly, the fourth component is the HLA (high-level architecture)-based integrated simulation interface for the convenient and efficient exchange of the data through the middleware. To show the applicability of the proposed method, it has been applied to a block turn-over simulation for which one floating crane and two crawler cranes were used, and an offshore module installation for which a DCR (dual-crane rig) was used. In conclusion, the execution of the proposed method of this study is successful regarding the above two applications for which multiple workers were involved.

Dynamic Simulation of a Shipbuilding Erection Crane based on Wire Rope Dynamics (Wire Rope Dynamics 기반의 조선용 탑재 크레인 동역학 시뮬레이션)

  • Cha, Ju-Hwan;Ku, Nam-Kug;Roh, Myung-Il;Lee, Kyu-Yeul
    • Journal of the Computational Structural Engineering Institute of Korea
    • /
    • v.25 no.2
    • /
    • pp.119-127
    • /
    • 2012
  • A wire rope is comprised of several metal wires which are wound together like a helix and it can resist relatively large axial loads, as compared with bending and torsional loads. A shipbuilding crane for erection such as a floating crane, a gantry crane, and a crawler crane hoists up and down heavy blocks by using these wire ropes. Thus, it is necessary to find dynamic properties of a wire rope in order to safely lift the blocks using the crane. In this study, a formula for calculating the tension and torsional moment acting on wire ropes of the crane was derived based on the existing study, and then dynamic simulation of the crane was performed based on the formula. The result shows that the dynamic simulation can be applied to find the safe method for block erection of shipyards.

A Study on the Hyperlink Network Analysis of Library Web Sites (도서관 웹사이트의 하이퍼링크 네트워크 분석)

  • Roh, Yoon-Ju;Kim, Seong-Hee
    • Journal of the Korean BIBLIA Society for library and Information Science
    • /
    • v.28 no.2
    • /
    • pp.99-117
    • /
    • 2017
  • The present study positively analyzed the hyperlinks of 32 web sites with the purpose of analyzing the hyperlink network structure of web sites for each domestic library type. After collecting the hyperlink data using the crawler, we analyzed the overall characteristics of the websites in the network based on the characteristics of the library. The results are as follows. 1) Among all analyzed libraries, Yonsei scored the highest in degree centrality, betweenness centrality, closeness centrality, and eigenvector centrality. 2) By library type, Sejong for national library, Seoul for public library, and Yonsei for college library appeared an influential a relatively. Based on these analysis results, the present study will be utilized as basic data for establishing an operation strategy that improves the efficiency and effectiveness of library web sites in the future.