• Title/Summary/Keyword: web data

Search Result 5,521, Processing Time 0.036 seconds

Web-server accelerator - CDA(Contents Delivery Accelerator) (웹 서버 성능 가속기)

  • 조준우;최현진;박규호
    • Proceedings of the IEEK Conference
    • /
    • 2003.11b
    • /
    • pp.73-76
    • /
    • 2003
  • Current web-server deals a multimedia data as well as text data. But dealing a multimedia data is high burden to web-server. So it can degrade web-server response. We introduce H/W feature CDA (Contents Delivery Accelerator). Main function of this H/W is transferring data between SCSI disk and NIC by direct path, and TCP offloading. These 2 functions can accelerate web-server performance. In this paper we will explain problem of current web-server and suggest our new architecture and say various implementation issues.

  • PDF

A Design of the Active Web Server Supporting Synchronous Collaboration in the Web-Based Groupware Systems (동기화된 협동을 지원하기 위한 능동형 웹 서버 설계)

  • 허순영;배경일
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.24 no.4
    • /
    • pp.157-170
    • /
    • 1999
  • The web-based groupware systems hold many possibilities for system developers and users. Especially, web-based group collaborative systems are emerging as enterprise-wide information systems. Since data in group collaborative systems are apt to be shared among multiple concurrent users and modified simutaneously by them, the web-based group collaborative systems must support synchronous collaboration in order to provide users with synchronized and consistent views of shared data. However, current web technologies have limitations in supporting this, largely because the existing Hypertext Transfer Protocol(HTTP) is unidirectional and does not allow web servers to send messages to their web browsers without first receiving requests from them. This paper proposes an active web server that can overcome such limitations and facilitate synchronous collaboration in web-based group collaborative systems. To accomplish such goals, the active web server manages dependency relationships beween shared data and web browsers referencing them and actively propagates changing details of the shared data to all web browsers referencing them. And, this paper examines usefulness and effectiveness of the active web server to apply it to the ball-bearing design example of concurrent engineering design systems. The prototype system of the active web server is developed on a commercial Object-oriented Database Management System(0DBMS) called OBJECTSTERE using the C++ programming language.

  • PDF

Refresh Cycle Optimization for Web Crawlers (웹크롤러의 수집주기 최적화)

  • Cho, Wan-Sup;Lee, Jeong-Eun;Choi, Chi-Hwan
    • The Journal of the Korea Contents Association
    • /
    • v.13 no.6
    • /
    • pp.30-39
    • /
    • 2013
  • Web crawler should maintain fresh data with minimum server overhead for large amount of data in the web sites. The overhead in the server increases rapidly as the amount of data is exploding as in the big data era. The amount of web information is increasing rapidly with advanced wireless networks and emergence of diverse smart devices. Furthermore, the information is continuously being produced and updated in anywhere and anytime by means of easy web platforms, and smart devices. Now, it is becoming a hot issue how frequently updated web data has to be refreshed in data collection and integration. In this paper, we propose dynamic web-data crawling methods, which include sensitive checking of web site changes, and dynamic retrieving of web pages from target web sites based on historical update patterns. Furthermore, we implemented a Java-based web crawling application and compared efficiency between conventional static approaches and our dynamic one. Our experiment results showed 46.2% overhead benefits with more fresh data compared to the static crawling methods.

A Web Based Training Service for Product Data Management (웹 기반 제품정보관리 교육 서비스)

  • Do N. C.
    • Korean Journal of Computational Design and Engineering
    • /
    • v.9 no.3
    • /
    • pp.260-265
    • /
    • 2004
  • This paper proposed a Web-based training service for product data management by supporting an integrated product data management system, various technical documents. and efficient communication systems. It also supports a general product development process and a consistent product data model that enable participants to experience management of consistent product information during the product development life cycle. The Web based environment of the service also provides participants with a collaborative workplace with other participants and a Web portal for all the components of the service.

Interoperability of Spatial Data through Open Web Map Server

  • Cho, D.S.;Jang, I.S.;Min, K.W.;Park, J.H.
    • Proceedings of the KSRS Conference
    • /
    • 2003.11a
    • /
    • pp.488-490
    • /
    • 2003
  • Over the past few years, a number of studies have been made on web map services, which enable the GIS user to access spatial data through the web. They, however, have focused on only implementation techniques separately, such as map client implementation, map server implementation, communication between client and server, and the map data representation. Therefore, it is hard to share and practical use the spatial data, because they does not ensure interoperability in heterogeneous map servers. In this paper, we have designed and implemented the web map server with open architecture, which complies with the standard interfaces proposed by OpenGIS Consortium (OGC). In particular, we have extended the OGC’s interfaces for a map server to support one or more data sources. This paper has contributed to construction and practical use of web map services by newly proposing the method of implementation of a map server, which could be reused regardless of the types of data sources.

  • PDF

Design and Implementation of a Web Crawler System for Collection of Structured and Unstructured Data (정형 및 비정형 데이터 수집을 위한 웹 크롤러 시스템 설계 및 구현)

  • Bae, Seong Won;Lee, Hyun Dong;Cho, DaeSoo
    • Journal of Korea Multimedia Society
    • /
    • v.21 no.2
    • /
    • pp.199-209
    • /
    • 2018
  • Recently, services provided to consumers are increasingly being combined with big data such as low-priced shopping, customized advertisement, and product recommendation. With the increasing importance of big data, the web crawler that collects data from the web has also become important. However, there are two problems with existing web crawlers. First, if the URL is hidden from the link, it can not be accessed by the URL. The second is the inefficiency of fetching more data than the user wants. Therefore, in this paper, through the Casper.js which can control the DOM in the headless brwoser, DOM event is generated by accessing the URL to the hidden link. We also propose an intelligent web crawler system that allows users to make steps to fine-tune both Structured and unstructured data to bring only the data they want. Finally, we show the superiority of the proposed crawler system through the performance evaluation results of the existing web crawler and the proposed web crawler.

A proposal on a proactive crawling approach with analysis of state-of-the-art web crawling algorithms (최신 웹 크롤링 알고리즘 분석 및 선제적인 크롤링 기법 제안)

  • Na, Chul-Won;On, Byung-Won
    • Journal of Internet Computing and Services
    • /
    • v.20 no.3
    • /
    • pp.43-59
    • /
    • 2019
  • Today, with the spread of smartphones and the development of social networking services, structured and unstructured big data have stored exponentially. If we analyze them well, we will get useful information to be able to predict data for the future. Large amounts of data need to be collected first in order to analyze big data. The web is repository where these data are most stored. However, because the data size is large, there are also many data that have information that is not needed as much as there are data that have useful information. This has made it important to collect data efficiently, where data with unnecessary information is filtered and only collected data with useful information. Web crawlers cannot download all pages due to some constraints such as network bandwidth, operational time, and data storage. This is why we should avoid visiting many pages that are not relevant to what we want and download only important pages as soon as possible. This paper seeks to help resolve the above issues. First, We introduce basic web-crawling algorithms. For each algorithm, the time-complexity and pros and cons are described, and compared and analyzed. Next, we introduce the state-of-the-art web crawling algorithms that have improved the shortcomings of the basic web crawling algorithms. In addition, recent research trends show that the web crawling algorithms with special purposes such as collecting sentiment words are actively studied. We will one of the introduce Sentiment-aware web crawling techniques that is a proactive web crawling technique as a study of web crawling algorithms with special purpose. The result showed that the larger the data are, the higher the performance is and the more space is saved.

A Study of Web Usage Mining for eCRM

  • Hyuncheol Kang;Jung, Byoung-Cheol
    • Communications for Statistical Applications and Methods
    • /
    • v.8 no.3
    • /
    • pp.831-840
    • /
    • 2001
  • In this study, We introduce the process of web usage mining, which has lately attracted considerable attention with the fast diffusion of world wide web, and explain the web log data, which Is the main subject of web usage mining. Also, we illustrate some real examples of analysis for web log data and look into practical application of web usage mining for eCRM.

  • PDF

Design and Implementation of Web-RTU Based on 8 bit MPU (8 비트 MPU 기반의 Web-RTU의 설계 및 구현)

  • Hong, Soon-Pil;Kim, Eun-Sung
    • Proceedings of the KIEE Conference
    • /
    • 2004.11c
    • /
    • pp.89-91
    • /
    • 2004
  • In SCADA systems, an RTU is a device installed at a remote location that collects data, codes the data into a format that is transmittable and transmits the data back to a central station, or MTU. An RTU also collects information from the master device and implements processes that are directed by master device. RTUs are equipped with input channels for sensing or metering, output channels for control, indication or alarms and a communications port. In general, the data are transmitted via a wired communication infrastructure such as RS422 or RS485 between RTU and MTU. But, limited range of wired communication doesn't allow the system to cover remote areas over the limitation, and building a wired communication network is not easy in the circumstances. In this Paper, we design and implement a smart cost-effective Web-RTU that can communicate with MTU via Web. Web is of benefit to the Web-RTU, because it is not only free from the distance limitations, but also is built easily and cost-effectively wherever Internet resources are available. Additionally, Web can be easily applied to the SCADA system with the development of hardware and software for communications. The Web-RTU has a program memory, a data memory and a RAM inside, and uses Atmega128, low-cost 8 bit micro-processor with eight AI(Analog Input). It performs well enough to implement all existing roles of RTU.

  • PDF

A Study of the Reliability of Web Services using Client Sides Errors

  • Lee, Sang-Bock;Kim, Mal-Suk
    • Journal of the Korean Data and Information Science Society
    • /
    • v.14 no.2
    • /
    • pp.217-221
    • /
    • 2003
  • Modeling the reliability of distributed systems requires a good understanding the reliability of the components. For thousands of web users, competitiveness in web services means a successful presence on the web. Failure rates for the presence of a web site are considered on client sides errors using RFC2068. Data were collected from some host via the internet.

  • PDF