• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2016.10a
• /
• pp.442-444
• /
• 2016

과거에 비해 비약적으로 생산되는 공간 데이터에 대한 처리를 위한 공간 연산은 빠른 처리 응답성을 요구하는 경우가 많다. 그래서 최근 하둡(Hadoop)과 같은 빅데이터 처리 시스템을 이용하여 처리하고자 하는 시도가 많다. 한편, 공간 조인은 데이터 분할(Partitioning)과 공간 색인의 이용 여부, 여과 단계와 정제 단계를 거치는 등 그 복잡도가 강한 공간 연산이다. 그래서 빅데이터 처리 시스템을 이용한 공간 조인의 처리 방식은 매우 다양하다. 그러나 지금까지 이러한 공간 조인의 처리 방식에 다른 리소스 활용에 대한 비교는 거의 없다. 이 논문에서는 다양한 공간 연산의 수행 방법에 따른 빅데이터 시스템 클러스터에서 데이터 전송 방식을 고찰하고 데이터 전송에 따른 네트워크 리소스의 효율적인 사용 방안을 제안하고자 한다. 구체적으로 단일할당과 다중할당 색인 기법의 비교, 파티셔닝 방법의 비교, 맵리듀스 시스템의 태스크 할당 방법에 따른 비교를 통해 다양한 연산 유형에 따른 공간 조인의 처리 방안 선정에 고려 요소를 제시하고자 한다.

• Journal of information and communication convergence engineering
• /
• v.20 no.2
• /
• pp.96-102
• /
• 2022

We aim to build predictive models for Airbnb's prices using a GPU-accelerated RAPIDS in a big data cluster. The Airbnb Listings datasets are used for the predictive analysis. Several machine-learning algorithms have been adopted to build models that predict the price of Airbnb listings. We compare the results of traditional and big data approaches to machine learning for price prediction and discuss the performance of the models. We built big data models using Databricks Spark Cluster, a distributed parallel computing system. Furthermore, we implemented models using multiple GPUs using RAPIDS in the spark cluster. The model was developed using the XGBoost algorithm, whereas other models were developed using traditional central processing unit (CPU)-based algorithms. This study compared all models in terms of accuracy metrics and computing time. We observed that the XGBoost model with RAPIDS using GPUs had the highest accuracy and computing time.

• Journal of the Chungcheong Mathematical Society
• /
• v.35 no.3
• /
• pp.205-225
• /
• 2022

In this paper, we derive analytical closed results for the first (a, b)-KA index, the Sombor index, the modified Sombor index, the first reduced (a, b)-KA index, the reduced Sombor index, the reduced modified Sombor index, the second reduced (a, b)-KA index and the mean Sombor index mSO_α for the OTIS biswapped networks by considering basis graphs as path, wheel graph, complete bipartite graph and r-regular graphs. Network theory plays a significant role in electronic and electrical engineering, such as signal processing, networking, communication theory, and so on. A topological index (TI) is a real number associated with graph networks that correlates chemical networks with a variety of physical and chemical properties as well as chemical reactivity. The Optical Transpose Interconnection System (OTIS) network has recently received increased interest due to its potential uses in parallel and distributed systems.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.21 no.6
• /
• pp.1183-1190
• /
• 2017

With the development of network and IT technology, people are searching and purchasing items they want, not bounded by places. Therefore, there are various studies on how to solve the scalability problem due to the rapidly increasing data in the recommendation system. In this paper, we propose an item-based collaborative filtering method using Tag weight and a recommendation technique using MapReduce method, which is a distributed parallel processing method. In order to improve speed and efficiency, the proposed method classifies items into categories in the preprocessing and groups according to the number of nodes. In each distributed node, data is processed by going through Map-Reduce step 4 times. In order to recommend better items to users, item tag weight is used in the similarity calculation. The experiment result indicated that the proposed method has been more enhanced the appropriacy compared to item-based method, and run efficiently on the large amounts of data.

• Journal of Digital Contents Society
• /
• v.18 no.7
• /
• pp.1341-1346
• /
• 2017

In this paper, we designed the open service platform based on integrated type of individual customized service and intelligent information technology with individual's complex attributes and requests. First, the data collection phase is proceed quickly and accurately to repeat extraction, transformation and loading. The generated data from extraction-transformation-loading process module is stored in the distributed data system. The data analysis phase is generated a variety of patterns that used the analysis algorithm in the field. The data processing phase is used distributed parallel processing to improve performance. The data providing should operate independently on device-specific management platform. It provides a type of the Open API.

• Journal of KIISE
• /
• v.43 no.4
• /
• pp.470-479
• /
• 2016

As the number of various types of media data such as UCC (User Created Contents) increases, research is actively being carried out in many different fields so as to provide meaningful media services. Amidst these studies, a semantic web-based media classification approach has been proposed; however, it encounters some limitations in video classification because of its underlying ontology derived from meta-information such as video tag and title. In this paper, we define recognized objects in a video and activity that is composed of video objects in a shot, and introduce a reasoning approach based on description logic. We define sequential rules for a sequence of shots in a video and describe how to classify it. For processing the large amount of increasing media data, we utilize Spark streaming, and a distributed in-memory big data processing framework, and describe how to classify media data in parallel. To evaluate the efficiency of the proposed approach, we conducted an experiment using a large amount of media ontology extracted from Youtube videos.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2003.09a
• /
• pp.640-643
• /
• 2003

A Walsh function based associative memory is capable of storing m patterns in a single pattern storage space with Walsh encoding of each pattern. Furthermore, each stored pattern can be matched against the stored patterns extremely fast using algorithmic parallel processing. As such, this special type of memory is ideal for real-time processing of large scale information. However this incredible efficiency generates large amount of crosstalk between stored patterns that incurs mis-recognition. This crosstalk is a function of the set of different sequencies [number of zero crossings] of the Walsh function associated with each pattern to be stored. This sequency set is thus optimized in this paper to minimize mis-recognition, as well as to maximize memory saying. In this paper, this Walsh memory has been applied to the problem of face recognition, where PCA is applied to dimensionality reduction. The maximum Walsh spectral component and genetic algorithm (GA) are applied to determine the optimal Walsh function set to be associated with the data to be stored. The experimental results indicate that the proposed methods provide a novel and robust technology to achieve an error-free, real-time, and memory-saving recognition of large scale patterns.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.5 no.6
• /
• pp.672-677
• /
• 2010

Recently, the efficient management system for large-scale sensor data has been required due to the increasing deployment of large-scale sensor networks. In this paper, we propose a cloud-based sensor data management system with low cast, high scalability, and efficiency. Sensor data in sensor networks are transmitted to the cloud through a cloud-gateway. At this point, outlier detection and event processing is performed. Transmitted sensor data are stored in the Hadoop HBase, distributed column-oriented database, and processed in parallel by query processing module designed as the MapReduce model. The proposed system can be work with the application of a variety of platforms, because processed results are provided through REST-based web service.

• The Journal of Information Technology
• /
• v.2 no.2
• /
• pp.145-161
• /
• 1999

This paper proposes a high speed back propagation neural networks which uses the residue number system. making the high speed operation possible without carry propagation Consisting of MAC(Multiplication and Accumulation) operator unit using Residue number system and sigmoid function operator unit using Mixed Residue Conversion is designed, The Designed circuits are descripted by VHDL and synthesized by Compass tools. Result of simulations shows that critical path delay time is about 19nsec and the size can be reduced to 40% compared to the neural networks implemented by the real number operation unit. The proposed design circuits can be implemented in parallel distributed processing system with desired real time processing.

• Journal of the Korea Society of Computer and Information
• /
• v.22 no.1
• /
• pp.49-56
• /
• 2017

In this paper we propose the approach for big data integration so as to analyze, visualize and predict the future of the trend of the market, and that is to get the integration data model using the R language which is the future of the statistics and the Hadoop which is a parallel processing for the data. As four approaching methods using R and Hadoop, ff package in R, R and Streaming as Hadoop utility, and Rhipe and RHadoop as R and Hadoop interface packages are used, and the strength and weakness of four methods are described and analyzed, so Rhipe and RHadoop are proposed as a complete set of data integration model. The integration of R, which is popular for processing statistical algorithm and Hadoop contains Distributed File System and resource management platform and can implement the MapReduce programming model gives us a new environment where in R code can be written and deployed in Hadoop without any data movement. This model allows us to predictive analysis with high performance and deep understand over the big data.