• Title/Summary/Keyword: Crawler

Search Result 200, Processing Time 0.022 seconds

A Study on Design and Development of Web Information Collection System Based Compare and Merge Method (웹 페이지 비교통합 기반의 정보 수집 시스템 설계 및 개발에 대한 연구)

  • Jang, Jin-Wook
    • Journal of Information Technology Services
    • /
    • v.13 no.1
    • /
    • pp.147-159
    • /
    • 2014
  • Recently, the quantity of information that is accessible from the Internet is being dramatically increased. Searching the Web for useful information has therefore become increasingly difficult. Thus, much research has been done on web robots which perform internet information filtering based on user interest. If a web site which users want to visit is found, its content is searched by following the searching list or Web sites links in order. This search process takes a long time according as the number of page or site increases so that its performance need to be improved. In order to minimize unnecessary search with web robots, this paper proposes an efficient information collection system based on compare and merge method. In the proposed system, a web robot initially collects information from web sites which users register. From the next visit to the web sites, the web robot compares what it collected with what the web sites have currently. If they are different, the web robot updates what it collected. Only updated web page information is classified according to subject and provided to users so that users can access the updated information quickly.

Impacts of Exploitation and Exploration on Performance of Open Collaboration: Focus on Open Source Software Development Project (지식의 탐색(Exploration)과 활용(Exploitation)이 개방형협업의 성과에 미치는 영향: 오픈소스 소프트웨어 개발 프로젝트를 중심으로)

  • Lee, Saerom;Baek, Hyeon-Mi;Jang, Jeong-Ju
    • Knowledge Management Research
    • /
    • v.18 no.2
    • /
    • pp.85-102
    • /
    • 2017
  • With rapid development of information and communication technologies, open collaboration can be eased through the Internet. Open source software, as a representative area of open collaboration, is developed and adopted to various fields. In this research, based on organizational learning theory, we examine the impacts of exploration and exploitation on innovation performance in open source software development projects. We define knowledge exploration as a number of developers from outside organization and knowledge exploitation as the ratio of member of an organization who participated in an open source software project managed by the organization. For analysis, we collect data of 4794 projects from github which is a representative open source software development platform using Web crawler developed by Python. As a result, we find that excessive exploration has curvilinear (invers U-shape) relationship on project performance. On the other hand, exploitation with enough external developers will positively impact on project performance.

Innovative Methodology for Assembling Jack up Leg of 205m on ground of Ultra

  • Yang, Yeong-Tae;Sim, Song-Seop;Lee, Seung-Yeop;Hwang, Oe-Ju;Sin, Bong-Yeong
    • Proceedings of the Korea Committee for Ocean Resources and Engineering Conference
    • /
    • 2003.10a
    • /
    • pp.87-92
    • /
    • 2003
  • Generally, in jack up rig design for harsh environment, its leg height is a major factor for achieving a sufficient serviceability & operability in terms of the worst environment and the workable depth. Due to difficulties in constructing such a high-slender leg, inaccessibility of yard fabrication equipment, etc. the construction of Jack up rig fur harsh deep sea has not been common. Method using heavy crawler crane, fabrication tower or extension by the floating crane vessel is still conventional construction but, considering high cost fur mobilizing heavy lift vessel (HLV) or additional marine work for implementing preload / full height test at sea, the ground-base construction is much advantageous. Air skidding method (ASM hereafter) is ground-based construction methodology, newly developed due to such requests. ASM could also be extended to similar engineering fields. This paper presents the operating sequence, design parameters and procedure which were verified through successful operation at the end of May 2002.

  • PDF

Development of Differentially Driven Inpipe Inspection Robot for Underground Gas Pipeline (지하 매설 가스배관용 차동 구동형 배관검사 로봇의 개발)

  • No, Se-Gon;Ryu, Seong-Mu;Choe, Hyeok-Ryeol
    • Transactions of the Korean Society of Mechanical Engineers A
    • /
    • v.25 no.12
    • /
    • pp.2019-2029
    • /
    • 2001
  • Up to now a wide variety of researches on inpipe inspection robots have been introduced, but it still seems to be difficult to construct a robot providing mobility sufficient to navigate inside the complicated configuration of underground pipelines. This paper introduces a robot called MRINSPECT IV(Multifunctional Robotic Crawler for inpipe inSPECTion IV) for the inspection of urban gas pipelines with a nominal 4-inch inside diameter. The proposed robot can freely move along the basic configuration of pipelines such as along horizontal or vertical pipelines. Moreover it can travel along reducers, elbows, and steer in the branches by modulating the speeds of driving modules. Especially, its capability for steering in tile three-dimensional pipeline configuration has a competative edge over the other ones and provides excellent mobility in navigation. Its critical points in the design and construction are introduced and results of experiments are given.

UNSM Surface Technology for Manufacturing and Remanufacturing Torsion Bars for Crawler Vehicles (초음파 나노표면개질을 적용한 궤도차량용 토션바 제조 및 재제조용 표면 개질기술에 관한 연구)

  • Suh, Chang-Min;Pyoun, Young-Sik;Cho, In-Ho;Baek, Un-Bong
    • Journal of Ocean Engineering and Technology
    • /
    • v.25 no.6
    • /
    • pp.80-85
    • /
    • 2011
  • The Ultrasonic Nanocrystal Surface Modification (UNSM) technology improves the fatigue life of a torsion bar by inducing compressive residual stress on the surface layer. The UNSM is applied to replace the presetting method and shot peening technology. The torsion bar must be changed periodically because of a lack of durability and a phenomenon related to the stress relaxation. The torsion fatigue test specimens were made of DIN17221 material, and the results showed that the fatigue life was 5 times more than under durability test conditions. A comparison test between the commercial vehicles' presetting method and shot peened torsion bar and the UNSM torsion bar showed that the UNSM could replace the presetting method and shot peening.

A Study of E-mail and Personal Homepage as a Marketing Promotion Tool in the Hotel Industry (호텔에서 마케팅 도구로써 이메일과 개인 홈페이지의 활용방안에 관한 연구)

  • Chung Hyun-Young
    • The Journal of the Korea Contents Association
    • /
    • v.4 no.4
    • /
    • pp.11-19
    • /
    • 2004
  • With the help of information technology the number of email users and personal home page owners are increasing. Marketers have much interest in using the email and personal home pages as a marketing promotion tool which can provide potential customers with messages they want to send. Marketers can facilitate the promotion efforts once if the profiles of potential customers' information can be databased by sending proper messages to the targeted market. Because of the merit of email and personal home page hotel firms are expected to adopt the information applications in their promotions for customers. This study proposes the Possibilities of email and personal home pages as a marketing promotion tool in the hotel industry and discusses problems to be overcome.

  • PDF

System Design for Collecting Real-Time Product Information Using RSS (RSS를 이용한 실시간 상품정보 수집시스템의 설계)

  • Chuluun, Munkhzaya;Ko, Sun-Woo
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.35 no.1
    • /
    • pp.1-9
    • /
    • 2012
  • It is well known that internet shoppers are very sensitive to sale prices. They visit the various shopping malls and collect the product information including purchase conditions for goods purchase decision-making. Recently the necessity of information support is increasing because of increase of information amount which is necessary and complexity of goods purchase decision-making process. The comparison shopping agent systems have provided price comparison information which is collected from various shopping malls to satisfy internet shoppers information craving. But the frequent price change caused by keen price competition is becoming the primary reason of information quality decline among price comparison sites. RSS which is a family of web feed formats used to publish frequently updated is applied even in on-line shopping malls. This paper develops a RSS product information collection system to get real-time product information. The proposed product information system consists of (1) web crawler module for searching RSS feed shopping malls automatically, (2) RSS reader module for parsing product information from RSS feed file, (3) product DB and (4) product searching module. Performance of the proposed system is higher than the comparison shopping agent systems when it is defined with the volume of collecting product information per unit time.

Extended Kalman Filter-based Localization with Kinematic Relationship of Underwater Structure Inspection Robots (수중 구조물 검사로봇의 기구학적 관계를 이용한 확장 칼만 필터 기반의 위치추정)

  • Heo, Young-Jin;Lee, Gi-Hyeon;Kim, Jinhyun
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.19 no.4
    • /
    • pp.372-378
    • /
    • 2013
  • In this paper, we research the localization problem of the crawler-type inspection robot for underwater structure which travels an outer wall of underwater structure. Since various factors of the underwater environment affect an encoder odometer, it is hard to localize robot itself using only on-board sensors. So in this research we used a depth sensor and an IMU to compensate odometer which has extreme error in the underwater environment through using Extended Kalman Filter(EKF) which is normally used in mobile robotics. To acquire valid measurements, we implemented precision sensor modeling after assuming specific situation that robot travels underwater structure. The depth sensor acquires a vertical position of robot and compensates one of the robot pose, and IMU is used to compensate a bearing. But horizontal position of robot can't be compensated by using only on-board sensors. So we proposed a localization algorithm which makes horizontal direction error bounded by using kinematics relationship. Also we implemented computer simulations and experiments in underwater environment to verify the algorithm performance.

Analyzing Box-Office Hit Factors Using Big Data: Focusing on Korean Films for the Last 5 Years

  • Hwang, Youngmee;Kim, Kwangsun;Kwon, Ohyoung;Moon, Ilyoung;Shin, Gangho;Ham, Jongho;Park, Jintae
    • Journal of information and communication convergence engineering
    • /
    • v.15 no.4
    • /
    • pp.217-226
    • /
    • 2017
  • Korea has the tenth largest film industry in the world; however, detailed analyses using the factors contributing to successful film commercialization have not been approached. Using big data, this paper analyzed both internal and external factors (including genre, release date, rating, and number of screenings) that contributed to the commercial success of Korea's top 10 ranking films in 2011-2015. The authors developed a WebCrawler to collect text data about each movie, implemented a Hadoop system for data storage, and classified the data using Map Reduce method. The results showed that the characteristic of "release date," followed closely by "rating" and "genre" were the most influential factors of success in the Korean film industry. The analysis in this study is considered groundwork for the development of software that can predict box-office performance.

Study of a Variable Single-tracked Crawler for Overcoming Obstacles (가변형 단일 궤도를 이용한 장애물 극복방법에 관한 연구)

  • Kim, Jee-Hong;Lee, Chang-Goo
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.16 no.4
    • /
    • pp.391-395
    • /
    • 2010
  • In our paper, we propose an asymmetric single-tracked wheel system, and describe its structure and the method for maintaining the length of a transformable track system. And the method is reducing the gap of lengths. Therefore, we propose an efficient structure for transforming and explain motions with kinematics. Our transformable shape single-tracked mobile system has an advantage to overcome an obstacle or stairs by the variable arms in the single unity track system. But we will make the variable shape of tracked system get a drive that has a force to stand against a wall. In this case, we can consider this system to a rigid body and have a notice that this single tracked system is able to get vary shape with the variable arm angle. Considering forces balance along x-axis and y-axis, and moments balance around the center of the mass we have. If this rigid body is standing against a wall and doesn't put in motion, the force of flat ground and the rigid body sets an equal by a friction. In the same way, the force of a wall and the rigid sets an equal by a friction.