• Proceedings of the KIEE Conference
• /
• 1975.08a
• /
• pp.99-101
• /
• 1975

• Optical Science and Technology
• /
• v.9 no.3
• /
• pp.23-31
• /
• 2005

• Communications of the Korean Institute of Information Scientists and Engineers
• /
• v.18 no.3
• /
• pp.28-39
• /
• 2000

• Journal of KIISE:Computing Practices and Letters
• /
• v.14 no.6
• /
• pp.567-581
• /
• 2008

As the size of the web is growing explosively, search engines are becoming increasingly important as the primary means to retrieve information from the Internet. A search engine periodically downloads web pages and stores them in the database to provide readers with up-to-date search results. The web crawler is a program that downloads and stores web pages for this purpose. A large-scale search engines uses a parallel web crawler to retrieve the collection of web pages maximizing the download rate. However, the service architecture or experimental analysis of parallel web crawlers has not been fully discussed in the literature. In this paper, we propose an architecture of the parallel web crawler and discuss implementation issues in detail. The proposed parallel web crawler is based on the coordinator/agent model using multiple machines to download web pages in parallel. The coordinator/agent model consists of multiple agent machines to collect web pages and a single coordinator machine to manage them. The parallel web crawler consists of three components: a crawling module for collecting web pages, a converting module for transforming the web pages into a database-friendly format, a ranking module for rating web pages based on their relative importance. We explain each component of the parallel web crawler and implementation methods in detail. Finally, we conduct extensive experiments to analyze the effectiveness of the parallel web crawler. The experimental results clarify the merit of our architecture in that the proposed parallel web crawler is scalable to the number of web pages to crawl and the number of machines used.

• Journal of KIISE
• /
• v.44 no.8
• /
• pp.791-802
• /
• 2017

Competition in the modern CPU market has shifted from speeding up the clock speed of a single core to increasing the number of cores. As such, there is a growing interest in parallel programming to maximize the use of resources of many core processors. In this paper, we propose parallel programming models in Haskell to find an advisable parallel programming model for many-core environments. Specifically, we used Eval monad and Cloud Haskell to develop two versions of parallel programs: plagiarism detection and K-means. Then, we evaluated the performance of the developed programs in 32-core and 120-core environments. The results of our experiment show that the Eval monad is highly efficient in an environment with a small number of cores. On the other hand, the Cloud Haskell runtime shows 37% improvement over Eval monad and the scalability shows a 134% improvement over Eval monad as the number of cores increases.

• Journal of KIISE
• /
• v.41 no.9
• /
• pp.611-616
• /
• 2014

Rabin fingerprinting used for chunking requires the largest amount computation time in data deduplication, In this paper, therefore, we proposed parallel Rabin fingerprinting on GPGPU for efficient data deduplication. In addition, for efficient parallelism in Rabin fingerprinting, four issues are considered. Firstly, when dividing input data stream into data sections, we consider the data located near the boundaries between data sections to calculate Rabin fingerprint continuously. Secondly, we consider exploiting the characteristics of Rabin fingerprinting for efficient operation. Thirdly, we consider the chunk boundaries which can be changed compared to sequential Rabin fingerprinting when adapting parallel Rabin fingerprinting. Finally, we consider optimizing GPGPU memory access. Parallel Rabin fingerprinting on GPGPU shows 16 times and 5.3 times better performance compared to sequential Rabin fingerprinting on CPU and compared to parallel Rabin fingerprinting on CPU, respectively. These throughput improvement of Rabin fingerprinting can lead to total performance improvement of data deduplication.

• Journal of KIISE:Computing Practices and Letters
• /
• v.8 no.4
• /
• pp.411-422
• /
• 2002

Recently, there have been active researches about the VPCS (Virtual Parallel Computing System) based on multiple agents. The PVCS uses personal computers or workstations that are dispersed all over the internet, rather than a high-cost supercomputer, to solve complex problems that require a huge number of calculations. It can be made up with either homogeneous or heterogeneous computers, depending on resources available on the internet. In this paper, we propose a new method in order to distribute worker agents and work packages efficiently on the VPCS based on the IBM Aglets. The previous methods use mainly the master-slave pattern for distributing worker agents and work packages. However, in these methods the workload increases dramatically at the central master as the number of agents increases. As a solution to this problem, our method appoints worker agents to distribute worker agents and workload packages. The proposed method is evaluated in several ways on the VPCS, and its results are improved to be worthy of close attention as compared with the previous ones.

• Journal of KIISE:Computer Systems and Theory
• /
• v.30 no.10
• /
• pp.586-594
• /
• 2003

X-Match algorithm is a lossless compression algorithm suitable for hardware implementation owing to its simplicity. It can compress 32 bits per clock cycle and is suitable for real time compression. However, as the bus width increases 64-bit, the compression unit also need to increase. This paper proposes the cooperative parallel X-Match (X-MatchCP) algorithm, which improves the compression speed by performing the two X-Match algorithms in parallel. It searches the all dictionary for two words, combines the compression codes of two words generated by parallel X-Match compression and outputs the combined code while the previous parallel X-Match algorithm searches an individual dictionary. The compression ratio in X-MatchCP is almost the same as in X-Match. X-MatchCP algorithm is described and simulated by Verilog hardware description language.

• Journal of KIISE:Software and Applications
• /
• v.27 no.4
• /
• pp.448-462
• /
• 2000

Coordinate constructions in natural language pose a number of difficulties to natural language processing units, due to the increased complexity of syntactic analysis, the syntactic ambiguity of the involved lexical items, and the apparent deletion of predicates in various places. In this paper, we address the syntactic characteristics of the coordinate constructions in Korean from the viewpoint of constructing a competence grammar, and present a version of combinatory categorial grammar for the analysis of coordinate constructions in Korean. We also show how to utilize a unified lexicon in the proposed grammar formalism in deriving the sentential semantics and associated information structures as well, in order to capture the discourse functions of coordinate constructions in Korean. The presented analysis conforms to the common wisdom that coordinate constructions are utilized in language not simply to reduce multiple sentences to a single sentence, but also to convey the information of contrast. Finally, we provide an analysis of sample corpora for the frequency of coordinate constructions in Korean and discuss some problematic cases.

• Journal of KIISE:Computer Systems and Theory
• /
• v.30 no.7_8
• /
• pp.341-351
• /
• 2003

For debugging effectively the shared-memory programs with nested parallelism, it is important to detect efficiently the first races which incur non-deterministic executions of the programs. Previous on-the-fly technique detects the first races in two passes, and shows inefficiencies both in execution time and memory space because the size of an access history for each shared variable depends on the maximum parallelism of program. This paper proposes a new on-the-fly technique to detect the first races in two passes, which is constant in both the number of event comparisons and the space complexity on each access to shared variable because the size of an access history for each shared variable is a small constant. This technique therefore makes on-the-fly race detection more efficient and practical for debugging shared-memory programs with nested parallelism.