• Journal of the Korea Convergence Society
• /
• v.9 no.10
• /
• pp.13-19
• /
• 2018

MapReduce, the Hadoop's essential core technology, is most commonly used to process big data based on the Hadoop distributed file system. However, the existing MapReduce-based big data processing techniques have a feature of dividing and storing files in blocks predefined in the Hadoop distributed file system, thus wasting huge infrastructure resources. Therefore, in this paper, we propose an efficient MapReduce-based big data processing scheme. The proposed method enhances the storage efficiency of a big data infrastructure environment by converting and compressing the data to be processed into a data format in advance suitable for processing by MapReduce. In addition, the proposed method solves the problem of the data processing time delay arising from when implementing with focus on the storage efficiency.

• Journal of Digital Convergence
• /
• v.12 no.4
• /
• pp.1-9
• /
• 2014

This research describes strategies to promote the growth of the Big Data industry and the companies within the ecosystem. In doing so, we identify the roles and responsibilities of various objects of this ecosystem and Big Data concepts. We describe the five components of the Big Data ecosystem: governance, data holders, service users, service providers and infrastructure providers. Related to the Big Data industry, the paper discusses 13 business strategies between the five components in the ecosystem. These strategies directly respond to areas of research by the Big Data industry leading experts on its early development. These strategies focus on how companies can gain competitive advantages in a growing new business environment of Big Data. The strategy topics are as follows: 1) the government's long term policy, 2) building Big Data support centers, 3) policy support and improving the legal system, 4) improving the Privacy Act, 5) increasing the understanding of Big Data, 6) Big Data support excavation projects, 7) professional manpower education, 8) infrastructure system support, 9) data distribution and leverage support, 10) data quality management, 11) business support services development, 12) technology research and excavation, 13) strengthening the foundation of Big Data technology. Of the proposed strategies, establishing supportive government policies is essential to the successful growth of thee Big Data industry. This study fosters a better understanding of the Big Data ecosystem and its potential to increases the competitive advantage of companies.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.23 no.10
• /
• pp.1179-1187
• /
• 2019

In today's technology environment, most big data-based applications and solutions are based on real-time processing of streaming data. Real-time processing and analysis of big data streams plays an important role in the development of big data-based applications and solutions. In particular, in the maritime data processing environment, the necessity of developing a technology capable of rapidly processing and analyzing a large amount of real-time data due to the explosion of data is accelerating. Therefore, this paper analyzes the characteristics of NiFi, Kafka, and Druid as suitable open source among various open data technologies for processing big data, and provides the latest information on external linkage necessary for maritime service analysis in Korean e-Navigation service. To this end, we will lay the foundation for applying open data framework technology for real-time data processing.

• Journal of information and communication convergence engineering
• /
• v.16 no.2
• /
• pp.99-105
• /
• 2018

k-Nearest neighbor join (k-NN Join) is a computationally intensive algorithm that is designed to find k-nearest neighbors from a dataset S for every object in another dataset R. Most related studies on k-NN Join are based on single-computer operations. As the data dimensions and data volume increase, running the k-NN Join algorithm on a single computer cannot generate results quickly. To solve this scalability problem, we introduce the locality-sensitive hashing (LSH) k-NN Join algorithm implemented in Spark, an approach for high-dimensional big data. LSH is used to map similar data onto the same bucket, which can reduce the data search scope. In order to achieve parallel implementation of the algorithm on multiple computers, the Spark framework is used to accelerate the computation of distances between objects in a cluster. Results show that our proposed approach is fast and accurate for high-dimensional and big data.

• The Journal of the Korea Contents Association
• /
• v.20 no.6
• /
• pp.1-8
• /
• 2020

The use of text data in big data analytics has been increased. So, much research on methods for text data analysis has been performed. In this paper, we study Bayesian learning based on conjugate prior for analyzing keyword data extracted from text big data. Bayesian statistics provides learning process for updating parameters when new data is added to existing data. This is an efficient process in big data environment, because a large amount of data is created and added over time in big data platform. In order to show the performance and applicability of proposed method, we carry out a case study by analyzing the keyword data from real patent document data.

• International journal of advanced smart convergence
• /
• v.5 no.2
• /
• pp.38-46
• /
• 2016

With the continuous effort to make business management more efficient, companies have started to utilize smart workplaces and the incorporation of mobile devices. Furthermore, big data processing, using Database as a Service (DBaas), is also being researched for integration. Similarly. mobile cloud can be utilized to allow for data sharing among employees. In this paper, in order to solve the issue of efficiency in business management, a collaboration system for data sharing using mobile cloud environment is explored. The proposed system, looks to benefit the increased integration of environment and corporate public through use of standardized data, in a design capable of efficient integrated management system.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.34 no.2
• /
• pp.207-215
• /
• 2016

The demand for spatial data service technologies is increasing lately with the development of realistic 3D spatial information services and ICT (Information and Communication Technology). Research is being conducted on the real-time provision of spatial data services through a variety of mobile and Web-based contents. Big data or cloud computing can be presented as alternatives to the construction of spatial data for the effective use of large volumes of data. In this paper, the process of building HLM (Hyper Live Map) using multi-source data to acquire stereo CCTV and other various data is presented and a big data service architecture design is proposed for the use of flexible and scalable cloud computing to handle big data created by users through such media as social network services and black boxes. The provision of spatial data services in real time using big data and cloud computing will enable us to implement navigation systems, vehicle augmented reality, real-time 3D spatial information, and single picture based positioning above the single GPS level using low-cost image-based position recognition technology in the future. Furthermore, Big Data and Cloud Computing are also used for data collection and provision in U-City and Smart-City environment as well, and the big data service architecture will provide users with information in real time.

• Journal of the Korean Society for Marine Environment & Energy
• /
• v.20 no.2
• /
• pp.117-125
• /
• 2017

SNS Big-data Analysis means to find potential value from big data which has produced by the social media. In this paper, SNS Big-data has been analysed to find Korean concerns by using 24 key words from the marine and fisheries sector. Among 24 key words, seafood, shipping and Dokdo Island are the most mentioned ones. Some key words such as ocean policies and marine security that have less concerns have bess mentioned less. Also, key words that are led by government are mostly mentioned by news media, but key words that are led by private sector and have intimate relationship with people's lives are mostly mentioned by Blogs and Twitters. Therefore, reflecting close national concerns by SNS Big-data Analysis and especially resolving negative factors are the most significant part of the policy establishment. Also, differentiated promotion methods need to be prepared because the frequency of key words mentioned from each type of media are different.

• Journal of Korea Port Economic Association
• /
• v.34 no.3
• /
• pp.159-182
• /
• 2018

The purpose of this study is to increase the competitiveness of big data in the maritime port organization, by understanding the expected performance and the intention to accept and use big data. In the empirical analysis of factors affecting the intention to use the big data technology for maritime port organizations, the variables employed are based on the Technology Organization Environment(TOE) and Diffusion of Innovations(DOI) theories, which are related to the acceptance of information and communication technologies. To achieve the objective of this study, an empirical analysis was conducted; this analysis targeted the personnel involved in the department of strategic planning and information technology in the related field. We set up eight hypotheses to examine the relevance between variables having three characteristics-technology, organization, and environmental characteristics. The empirical results are summarized as follows. First, it was seen that the technology characteristic, including relative advantage, complexity, and compatibility, has a significant effect on the expected performance. Second, the top management support of the organization characteristic has a significant effect, but the firm size of this characteristic has no significant effect on the expected performance. Third, the competitive pressure of the environment characteristic has a positive effect on the expected performance, while the regulatory support has no significant effect. Finally, the expected performance has a significant effect on the intention to use big data.

• The Journal of the Korea Contents Association
• /
• v.13 no.2
• /
• pp.211-219
• /
• 2013

The analysis method using Big Data has evolved based on the Big data Management Technology. There are quite a few researching institutions anticipating new era in data analysis using Big Data and IT vendors has been sided with them launching standardized technologies for Big Data management technologies. Big Data is also affected by improvements of IT gadgets IT environment. Foreran by social media, analyzing method of unstructured data is being developed focusing on diversity of analyzing method, anticipation and optimization. In the past, data analyzing methods were confined to the optimization of structured data through data mining, OLAP, statics analysis. This data analysis was solely used for decision making for Chief Officers. In the new era of data analysis, however, are evolutions in various aspects of technologies; the diversity in analyzing method using new paradigm and the new data analysis experts and so forth. In addition, new patterns of data analysis will be found with the development of high performance computing environment and Big Data management techniques. Accordingly, this paper is dedicated to define the possible analyzing method of social media using Big Data. this paper is proposed practical use analysis for social media analysis through data mining analysis methodology.