• 한국정보과학회논문지:컴퓨팅의 실제 및 레터
• /
• 제14권6호
• /
• pp.567-581
• /
• 2008

웹의 크기가 폭발적으로 증가함에 따라 인터넷에서 정보를 얻는 수단으로서 검색 엔진의 중요성이 부각되고 있다. 검색 엔진은 사용자에게 최신의 정보를 검색 결과로서 제공하기 위해 웹 페이지를 주기적으로 수집하고 이를 데이타베이스에 저장한다. 웹 크롤러는 이러한 목적으로 웹 페이지를 수집하는 프로그램이다. 대부분의 검색 엔진은 제한된 시간 내에 많은 수의 웹 페이지를 수집하기 위해 다수의 머신을 사용하는 병렬 웹 크롤러를 이용한다. 그러나, 병렬 웹 크롤러의 아키텍처와 세부 구현 방법이 잘 알려져 있지 않기 때문에 실제로 병렬 웹 크롤러를 구현하는 데에 어려움이 많다. 본 논문에서는 병렬 웹 크롤러(parallel web crawler)의 아키텍처와 세부 구현 방법을 제시한다. 병렬 웹 크롤러는 다수의 머신에서 웹 페이지를 병렬적으로 수집하기 위해 조정자(coordinator) 대리자(agent) 구조의 2-티어(tier) 모델을 사용한다. 조정자/대리자 모델은 각 머신에서 웹 페이지를 수집하기 위한 다수의 대리자들과 이 대리자들을 관리하기 위한 하나의 조정자로 구성된다. 병렬 웹 크롤러는 웹 페이지를 수집하기 위한 크롤링(crawling) 모듈, 수집한 웹 페이지를 데이타베이스 로딩 포맷으로 변환하기 위한 컨버팅(converting) 모듈, 수집된 웹 페이지의 중요도를 계산하기 위한 랭킹(ranking) 모듈로 구성된다. 본 논문에서는 병렬 웹 크롤러의 각 모듈들을 설명하고, 세부 구현 방법을 설명한다. 마지막으로, 실험을 통해 병렬 웹 크롤러의 성능을 평가하였다. 실험 결과, 제안된 병렬, 웹 크롤러가 수집해야할 웹 페이지 개수와 머신 개수에 따라 확장 가능함을 보였다.

• International Journal of Internet, Broadcasting and Communication
• /
• 제4권1호
• /
• pp.13-17
• /
• 2012

Indexing allows converting raw document collection into easily searchable representation. Web searching by Google or Yahoo provides subsecond response time which is made possible by efficient indexing of web-pages over the entire Web. Indexing process gets challenging when the scale gets bigger. Parallel techniques, such as MapReduce framework can assist in efficient large-scale indexing process. In this paper we propose PDFindexer, system for indexing scientific papers in PDF using MapReduce programming model. Unlike Web search engines, our target domain is scientific papers, which has pre-defined structure, such as title, abstract, sections, references. Our proposed system enables parsing scientific papers in PDF recreating their structure and performing efficient distributed indexing with MapReduce framework in a cluster of nodes. We provide the overview of the system, their components and interactions among them. We discuss some issues related with the design of the system and usage of MapReduce in parsing and indexing of large document collection.

• Journal of Information Science Theory and Practice
• /
• 제5권4호
• /
• pp.6-25
• /
• 2017

In this paper, a large-scale study on users' understanding of search-based advertising is presented. It is based on (1) a survey, (2) a task-based user study, and (3) an online experiment. Data were collected from 1,000 users representative of the German online population. Findings show that users generally lack an understanding of Google's business model and the workings of search-based advertising. 42% of users self-report that they either do not know that it is possible to pay Google for preferred listings for one's company on the SERPs or do not know how to distinguish between organic results and ads. In the task-based user study, we found that only 1.3 percent of participants were able to mark all areas correctly. 9.6 percent had all their identifications correct but did not mark all results they were required to mark. For none of the screenshots given were more than 35% of users able to mark all areas correctly. In the experiment, we found that users who are not able to distinguish between the two results types choose ads around twice as often as users who can recognize the ads. The implications are that models of search engine advertising and of information seeking need to be amended, and that there is a severe need for regulating search-based advertising.

• 한국시뮬레이션학회논문지
• /
• 제17권4호
• /
• pp.229-239
• /
• 2008

분산 파일 시스템(DFS)은 분산 환경에서 장애와 사본에 대한 투명성을 보장하며 파일을 다수의 물리적인 컴퓨터 노드들에게 저장할 수 있는 메카니즘을 제공한다. 검색엔진, 그리드 컴퓨팅, 데이터 마이닝 어플리케이션등과 같이 많은 양의 데이터를 처리하는 어플리케이션들은 데이터 저장을 위한 백엔드 인프라 구조를 제공할 필요가 있다. 분산 파일 시스템은 이러한 저장 데이터 기반을 위한 주요 구성요소가 된다. 많은 프로젝트의 관심사가 되는 네트워크 컴퓨팅은 이와 같이 설계 및 구현된 분산파일 시스템을 갖추고 있으며, 다양한 아키텍처와 기능들을 시스템의 특성에 따라서 제공하고 있다. 이 논문에서는 대용량의 검색 지향적인 시스템에서 사용되는 SOFA 분산 파일 시스템, 메카니즘들과 성능들을 소개한다.

• International Journal of Computer Science & Network Security
• /
• 제22권2호
• /
• pp.406-412
• /
• 2022

Storing large amounts of data has always been a big problem from the beginning of computing history. Big Data has made huge advancements in improving business processes by finding the customers' needs using prediction models based on web and social media search. The main purpose of big data stream processing frameworks is to allow programmers to directly query the continuous stream without dealing with the lower-level mechanisms. In other words, programmers write the code to process streams using these runtime libraries (also called Stream Processing Engines). This is achieved by taking large volumes of data and analyzing them using Big Data frameworks. Streaming platforms are an emerging technology that deals with continuous streams of data. There are several streaming platforms of Big Data freely available on the Internet. However, selecting the most appropriate one is not easy for programmers. In this paper, we present a detailed description of two of the state-of-the-art and most popular streaming frameworks: Apache Ignite and Hazelcast. In addition, the performance of these frameworks is compared using selected attributes. Different types of databases are used in common to store the data. To process the data in real-time continuously, data streaming technologies are developed. With the development of today's large-scale distributed applications handling tons of data, these databases are not viable. Consequently, Big Data is introduced to store, process, and analyze data at a fast speed and also to deal with big users and data growth day by day.

• 대한한방부인과학회지
• /
• 제35권1호
• /
• pp.34-58
• /
• 2022

Objectives: This study was performed to analyze the interventions of Oriental Medicine which had been commonly used for postpartum disease and postpartum care. Methods: We searched research on the interventions for postpartum disease and postpartum care in 4 domestic search engines. After that, we conducted eligibility screening based on inclusion and exclusion criteria. Results: 1. We selected total 50 studies. There were 2 randomized controlled trial (RCT), 5 non-RCT, 35 case reports, 8 case series within the 6~8 weeks after childbirth. 2. Of the 35 case reports, several interventions were used : acupuncture (22), moxibustion (11), cupping therapy (7), pharmacopuncture (5), chuna manipulation (4), herbal medicine (34). The most common symptoms were musculoskeletal symptoms (8), followed by postpartum depression (7). Various prescriptions and acupoints of oriental medicine were used depending on the diseases or symptoms. 3. Of the 8 case series, 382 subjects in 5 case series had taken Saenghwa-tang-gagam. And Acupuncture, moxibustion, cupping therapy (5), pharmacopuncture (1) were used as an intervention. 4. The most commonly used acupoint is 腎兪 (BL23) in the pain including postpartum back pain and 三陰交 (SP6), 關元 (CV4) in the postpartum care. 關元 (CV4) is the most commomly used moxibustion point not only the postpartum disease but also the postpartum care. Conclusions: In clinical studies of oriental medicine related to postpartum disease and postpartum care, pain-related clinical studies that belong to or progress to Sanhupung were the most common (30%), and among them, postpartum low back pain studies were the most common (20%). Based on this, we believe that large-scale clinical studies with high quality using oriental interventions including chuna and pharmacopuncture are needed to establish guidelines for the management of pain treatment including postpartum back pain.