• Journal of KIISE
• /
• v.43 no.4
• /
• pp.450-459
• /
• 2016

Every year, RDFS data tends further toward scalability; hence, the manner of SPARQL processing needs to be changed for fast query. The query processing method of SPARQL has been studied using a scalable distributed processing framework. Current studies indicate that the query engine based on the scalable distributed processing framework i.e., Hadoop(MapReduce) is not suitable for real-time processing because of the repetitive tasks; in addition, it is difficult to construct a query engine based on an In-memory Distributed Query engine, because distributed structure on the low-level is required to be considered. In this paper, we proposed a method to construct a query engine for improving the speed of the query process with the mass triple data. The query engine processes the query of SPARQL using the SparkSQL, which is an In-memory based, distributed query processing framework. SparkSQL is a high-level distributed query engine that facilitates existing SQL statement. In order to process the SPARQL query, after generating the Algebra Tree using Jena, the Algebra Tree is required to be translated to Spark Algebra Tree for application in the Spark system, and construction of the system that generated the SparkSQL query. Furthermore, we proposed the design of triple property table based on DataFrame for more efficient query processing in the Spark system. Finally, we verified the validity through comparative evaluation with the query engine, which is the existing distributed processing framework.

• Journal of the Korea Society of Computer and Information
• /
• v.27 no.2
• /
• pp.171-177
• /
• 2022

In this paper, we propose a service method that can provide insight into multi-source agricultural data, way to cluster environmental factor which supports data analysis according to time flow, and curate crop environmental factors. The proposed curation service consists of four steps: collection, preprocessing, storage, and analysis. First, in the collection step, the service system collects and organizes multi-source agricultural data by using an OpenAPI-based web crawler. Second, in the preprocessing step, the system performs data smoothing to reduce the data measurement errors. Here, we adopt the smoothing method for each type of facility in consideration of the error rate according to facility characteristics such as greenhouses and open fields. Third, in the storage step, an agricultural data integration schema and Hadoop HDFS-based storage structure are proposed for large-scale agricultural data. Finally, in the analysis step, the service system performs DTW-based time series classification in consideration of the characteristics of agricultural digital data. Through the DTW-based classification, the accuracy of prediction results is improved by reflecting the characteristics of time series data without any loss. As a future work, we plan to implement the proposed service method and apply it to the smart farm greenhouse for testing and verification.

• Journal of the Korean Society of Manufacturing Process Engineers
• /
• v.17 no.2
• /
• pp.1-7
• /
• 2018

Recently, as interest in the internet of things has increased, a vibration energy harvester has attracted attention as a power supply method for a wireless sensor. The vibration energy harvester can be divided into piezoelectric types, electromagnetic type and electrostatic type, according to the energy conversion type. The electromagnetic vibration energy harvester has advantages, in terms of output density and design flexibility, compared to other methods. The efficiency of an electromagnetic vibration energy harvester is determined by the shape, size, and spacing of coils and magnets. Generating all the experimental cases is expensive, in terms of time and money. This study proposes a method to perform design optimization of an electromagnetic vibration energy harvester using an open source, big data platform.

• Proceedings of the Korean Information Science Society Conference
• /
• 2011.06b
• /
• pp.377-380
• /
• 2011

음악은 인류의 대표적인 예술로서 오랜 세월동안 사랑을 받아왔다. 그 오래된 세월만큼이나 인류가 만들어온 음악의 수는 방대하다. 방대한 음악이 IT기술의 발달과 인터넷의 확산을 통하여 온라인 음악시장을 형성하였고 음악은 디지털 음원으로 관리되게 되었다. 이러한 디지털 음원을 효과적으로 검색하기 위한 방법은 많이 연구되었다. 그리고 검색을 도와줄 대량의 디지털 음원 자료들을 저장하고 관리하는 기법에 관한 연구가 필요하다. 본 논문에서는 대용량 자료를 처리하는 기술로 관심 받고 있는 하둡을 통하여 이 문제를 연구하였다. 하둡의 맵리듀스, HDFS 그리고 HBase를 이용하여 음악 내용기반검색을 설계하였다. 본 시스템은 음악 검색 시스템을 관리하고 유지하는데 있어서 컴퓨팅자원을 절약함으로써 비용을 절감 효과를 얻을 수 있다.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.11 no.4
• /
• pp.433-440
• /
• 2016

In order to construct a system for big data processing, one needs to configure the node by using network equipments to connect multiple computers or establish cloud environments through virtual hosts on a single computer. However, there are many restrictions on constructing the big data analysis system including complex system configuration and cost. These constraints are becoming a major obstacle to professional manpower training for big data areas which is emerging as one of the most important national competitiveness. As a result, for professional manpower training of big data areas, this paper proposes a Raspberry Pi Board based educational big data processing system which is capable of practical training at an affordable price.

• Journal of Korea Multimedia Society
• /
• v.19 no.5
• /
• pp.808-817
• /
• 2016

Recently, the wearable computing technology with bio-sensors has been rapidly developed and utilized in various areas such as personal health, care-giving for senior citizens who live alone, and sports activities. In particular, the wearable computing equipment to measure vital signs by means of digital yarns and bio sensors is noticeable. The wearable computing devices help users monitor and manage their health in their daily lives through the customized healthcare service. In this paper, we suggest a system for monitoring and analyzing vital signs utilizing smart healthcare clothing with bio-sensors. Vital signs that can be continuously acquired from the clothing is well-known as unstructured data. The amount of data is huge, and they are perceived as the big data. Vital sings are stored by Hadoop Distributed File System(HDFS), and one can build data warehouse for analyzing them in HDFS. We provide health monitoring system based on vital sings that are acquired by biosensors in smart healthcare clothing. We implemented a big data platform which provides health monitoring service to visualize and monitor clinical information and physical activities performed by the users.

• Journal of Korea Multimedia Society
• /
• v.17 no.6
• /
• pp.671-680
• /
• 2014

Recently, with the remarkable increase of social network services, it is necessary to extract interesting information from lots of data about various individual opinions and preferences on SNS(Social Network Service). The sentiment information can be applied to various fields of society such as politics, public opinions, economics, personal services and entertainments. To extract sentiment information, it is necessary to use processing techniques that store a large amount of SNS data, extract meaningful data from them, and search the sentiment information. This paper proposes an efficient method to extract sentiment information from various unstructured big data on social networks using HDFS(Hadoop Distributed File System) platform and MapReduce functions. In experiments, the proposed method collects and stacks data steadily as the number of data is increased. When the proposed functions are applied to sentiment analysis, the system keeps load balancing and the analysis results are very close to the results of manual work.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2017.04a
• /
• pp.429-432
• /
• 2017

자동차 보급률 증가로 인해 교통 혼잡, 불법 주정차 등의 사회적 문제가 발생하고 있다. 특히 불법 주정차는 교통 혼잡, 주차 공간 부족 등 부가적인 문제를 발생시키고 있다. 따라서 각 지방자치단체에서는 불법 주정차 문제를 해결하기 위한 방안을 연구하고 있다. 그러나 이러한 방안은 초기 비용 발생 및 인력 부족 등의 한계가 있다. 한편, 정보통신의 발달에 따라 공공 업무에도 대량의 공공데이터를 효율적으로 처리하기 위한 연구가 진행되고 있다. 하지만 이러한 연구 또한 빅데이터 처리 플랫폼 부족 및 분석 시스템이 미흡한 한계가 존재한다. 따라서 본 논문에서는 불법 주정차 데이터와 같은 공공 데이터를 효율적으로 처리하기 위해, 주정차 단속 시스템을 위한 하둡 기반 대용량 데이터 관리 및 분석 시스템을 제안한다. 제안하는 시스템은 첫째, 주차단속을 수행할 때 주차단속 데이터를 하이브(Hive)를 통해 저장하고, 단속된 차량의 차주를 검색하여 단속임을 알리거나 과태료를 부과한다. 둘째, 웹 인터페이스를 통해 수집된 주차단속 데이터에 대한 다양한 분석을 수행하고, 분석된 데이터에 대한 R을 이용한 시각화를 제공한다.

• The KIPS Transactions:PartD
• /
• v.18D no.2
• /
• pp.81-88
• /
• 2011

Sequential pattern mining that determines frequent patterns appearing in a given set of sequences is an important data mining problem with broad applications. For example, sequential pattern mining can find the web access patterns, customer's purchase patterns and DNA sequences related with specific disease. In this paper, we develop the sequential pattern mining algorithms using MapReduce framework. Our algorithms distribute input data to several machines and find frequent sequential patterns in parallel. With synthetic data sets, we did a comprehensive performance study with varying various parameters. Our experimental results show that linear speed up can be achieved through our algorithms with increasing the number of used machines.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2015.01a
• /
• pp.151-154
• /
• 2015

최근 소셜 네트워크, 바이오 컴퓨팅, 사물 인터넷 등의 출현으로 인해 기존의 IT환경보다 많은 데이터가 생성되고 있고, 이로 인해 효율적인 대용량 데이터 처리기법에 대한 연구가 진행 되고 있다. 맵리듀스는 데이터 집약적인 연산 어플리케이션에 효과적인 프로그래밍 모델로써, 대표적인 맵리듀스 어플리케이션으로는 아파치 소프트웨어 재단에서 개발 지원중인 하둡이 있다. 본 논문은 하둡 맵리듀스의 성능 향상을 위해 데이터 프리패칭 기법과 스트리밍 기법을 제안한다. 하둡 맵리듀스의 성능 이슈 중 하나는 맵리듀스 과정에서 입력 데이터 전송에 의한 작업 지연이다. 이러한 데이터 전송 시간을 최소화하기 위해, 기존 맵리듀스와는 달리 데이터 전송을 담당하는 프리패칭 스레드를 별도로 생성하였다. 그 결과 데이터의 맵리듀스 작업 중에도 데이터 전송이 가능하게 되어 전체 데이터 처리 시간을 줄일 수 있었다. 이러한 프리패칭 기법을 사용해도 하둡 맵리듀스의 특성상 최초 데이터 전송 시에는 작업대기를 하게 되는데, 이 대기시간을 줄이고자 스트리밍 기법을 사용하여 데이터 전송에 의한 대기시간을 추가로 줄일 수 있었다. 제안하는 기법의 성능을 측정하기 위해 수학적인 모델링을 하였으며, 성능 측정결과 기존의 하둡 맵리듀스 및 프리패칭 기법만 적용된 맵리듀스 보다 스트리밍 기법이 추가 적용된 맵리듀스의 성능이 향상되었음을 확인 할 수 있었다.