• Title/Summary/Keyword: BIG DATA

Search Result 6,154, Processing Time 0.033 seconds

Deduction of the Policy Issues for Activating the Geo-Spatial Big Data Services (공간 빅데이터 서비스 활성화를 위한 정책과제 도출)

  • Park, Joon Min;Lee, Myeong Ho;Shin, Dong Bin;Ahn, Jong Wook
    • Spatial Information Research
    • /
    • v.23 no.6
    • /
    • pp.19-29
    • /
    • 2015
  • This study was conducted with the purpose of suggesting the improvement plan of political for activating the Geo-Spatial Big Data Services. To this end, we were review the previous research for Geo-Spatial Big Data and analysis domestic and foreign Geo-Spatial Big Data propulsion system and policy enforcement situation. As a result, we have deduced the problem of insufficient policy of reaction for future Geo-Spatial Big Data, personal information protection and political basis service activation, relevant technology and policy, system for Geo-Spatial Big Data application and establishment, low leveled open government data and sharing system. In succession, we set up a policy direction for solving derived problems and deducted 5 policy issues : setting up a Geo-Spatial Big Data system, improving relevant legal system, developing technic related to Geo-Spatial Big Data, promoting business supporting Geo-Spatial Big Data, creating a convergence sharing system about public DB.

Automatic Generation of Issue Analysis Report Based on Social Big Data Mining (소셜 빅데이터 마이닝 기반 이슈 분석보고서 자동 생성)

  • Heo, Jeong;Lee, Chung Hee;Oh, Hyo Jung;Yoon, Yeo Chan;Kim, Hyun Ki;Jo, Yo Han;Ock, Cheol Young
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.3 no.12
    • /
    • pp.553-564
    • /
    • 2014
  • In this paper, we propose the system for automatic generation of issue analysis report based on social big data mining, with the purpose of resolving three problems of the previous technologies in a social media analysis and analytic report generation. Three problems are the isolation of analysis, the subjectivity of experts and the closure of information attributable to a high price. The system is comprised of the natural language query analysis, the issue analysis, the social big data analysis, the social big data correlation analysis and the automatic report generation. For the evaluation of report usefulness, we used a Likert scale and made two experts of big data analysis evaluate. The result shows that the quality of report is comparatively useful and reliable. Because of a low price of the report generation, the correlation analysis of social big data and the objectivity of social big data analysis, the proposed system will lead us to the popularization of social big data analysis.

A Business Application of the Business Intelligence and the Big Data Analytics (비즈니스 인텔리전스와 빅데이터 분석의 비즈니스 응용)

  • Lee, Ki-Kwang;Kim, Tae-Hwan
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.42 no.4
    • /
    • pp.84-90
    • /
    • 2019
  • Lately, there have been tremendous shifts in the business technology landscape. Advances in cloud technology and mobile applications have enabled businesses and IT users to interact in entirely new ways. One of the most rapidly growing technologies in this sphere is business intelligence, and associated concepts such as big data and data mining. BI is the collection of systems and products that have been implemented in various business practices, but not the information derived from the systems and products. On the other hand, big data has come to mean various things to different people. When comparing big data vs business intelligence, some people use the term big data when referring to the size of data, while others use the term in reference to specific approaches to analytics. As the volume of data grows, businesses will also ask more questions to better understand the data analytics process. As a result, the analysis team will have to keep up with the rising demands on the infrastructure that supports analytics applications brought by these additional requirements. It's also a good way to ascertain if we have built a valuable analysis system. Thus, Business Intelligence and Big Data technology can be adapted to the business' changing requirements, if they prove to be highly valuable to business environment.

Hadoop Based Wavelet Histogram for Big Data in Cloud

  • Kim, Jeong-Joon
    • Journal of Information Processing Systems
    • /
    • v.13 no.4
    • /
    • pp.668-676
    • /
    • 2017
  • Recently, the importance of big data has been emphasized with the development of smartphone, web/SNS. As a result, MapReduce, which can efficiently process big data, is receiving worldwide attention because of its excellent scalability and stability. Since big data has a large amount, fast creation speed, and various properties, it is more efficient to process big data summary information than big data itself. Wavelet histogram, which is a typical data summary information generation technique, can generate optimal data summary information that does not cause loss of information of original data. Therefore, a system applying a wavelet histogram generation technique based on MapReduce has been actively studied. However, existing research has a disadvantage in that the generation speed is slow because the wavelet histogram is generated through one or more MapReduce Jobs. And there is a high possibility that the error of the data restored by the wavelet histogram becomes large. However, since the wavelet histogram generation system based on the MapReduce developed in this paper generates the wavelet histogram through one MapReduce Job, the generation speed can be greatly increased. In addition, since the wavelet histogram is generated by adjusting the error boundary specified by the user, the error of the restored data can be adjusted from the wavelet histogram. Finally, we verified the efficiency of the wavelet histogram generation system developed in this paper through performance evaluation.

Implement of MapReduce-based Big Data Processing Scheme for Reducing Big Data Processing Delay Time and Store Data (빅데이터 처리시간 감소와 저장 효율성이 향상을 위한 맵리듀스 기반 빅데이터 처리 기법 구현)

  • Lee, Hyeopgeon;Kim, Young-Woon;Kim, Ki-Young
    • Journal of the Korea Convergence Society
    • /
    • v.9 no.10
    • /
    • pp.13-19
    • /
    • 2018
  • MapReduce, the Hadoop's essential core technology, is most commonly used to process big data based on the Hadoop distributed file system. However, the existing MapReduce-based big data processing techniques have a feature of dividing and storing files in blocks predefined in the Hadoop distributed file system, thus wasting huge infrastructure resources. Therefore, in this paper, we propose an efficient MapReduce-based big data processing scheme. The proposed method enhances the storage efficiency of a big data infrastructure environment by converting and compressing the data to be processed into a data format in advance suitable for processing by MapReduce. In addition, the proposed method solves the problem of the data processing time delay arising from when implementing with focus on the storage efficiency.

Modeling of Value Chain for Big Data (빅데이터를 위한 가치사슬 설계)

  • Lee, Sangwon;Park, Sungbum;Lee, Jumin;Ahn, Hyunsup;Choi, Yong Goo
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2015.01a
    • /
    • pp.277-278
    • /
    • 2015
  • The volume sub-challenge requires novel approaches, often referred to as Big Data technologies and methodologies. Data is generated constantly in an ever growing number of places and by an ever growing number of actors while a large proportion of potentially re-usable data resides within silos within institutions or companies. These are needed when conventional database technologies cannot be applied to storage and computing issues. The issue of big data has been referred to as the next frontier in computing. In this paper, we research on factors to design an organizational value chain for Big Data.

  • PDF

Utilization and Analysis of Big-data

  • Lee, Soowook;Han, Manyong
    • International Journal of Advanced Culture Technology
    • /
    • v.7 no.4
    • /
    • pp.255-259
    • /
    • 2019
  • This study reviews the analysis and characteristics of databases from big data and then establishes representational strategy. Thus, analysis has continued for a long time in the quantity and quality of data, and there are changes in the location of data in the social sciences, past trends and the emergence of big data. The introduction of big data is presented as a prototype of new social science and is a useful practical example that empirically shows the need, basis, and direction of analysis through trend prediction services. Big data provides a future perspective as an important foundation for social change within the framework of basic social sciences.

Information Visualization Process for Spatial Big Data (공간빅데이터를 위한 정보 시각화 방법)

  • Seo, Yang Mo;Kim, Won Kyun
    • Spatial Information Research
    • /
    • v.23 no.6
    • /
    • pp.109-116
    • /
    • 2015
  • In this study, define the concept of spatial big data and special feature of spatial big data, examine information visualization methodology for increase the insight into the data. Also presented problems and solutions in the visualization process. Spatial big data is defined as a result of quantitative expansion from spatial information and qualitative expansion from big data. Characteristics of spatial big data id defined as 6V (Volume, Variety, Velocity, Value, Veracity, Visualization), As the utilization and service aspects of spatial big data at issue, visualization of spatial big data has received attention for provide insight into the spatial big data to improve the data value. Methods of information visualization is organized in a variety of ways through Matthias, Ben, information design textbook, etc, but visualization of the spatial big data will go through the process of organizing data in the target because of the vast amounts of raw data, need to extract information from data for want delivered to user. The extracted information is used efficient visual representation of the characteristic, The large amounts of data representing visually can not provide accurate information to user, need to data reduction methods such as filtering, sampling, data binning, clustering.

Study of In-Memory based Hybrid Big Data Processing Scheme for Improve the Big Data Processing Rate (빅데이터 처리율 향상을 위한 인-메모리 기반 하이브리드 빅데이터 처리 기법 연구)

  • Lee, Hyeopgeon;Kim, Young-Woon;Kim, Ki-Young
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.12 no.2
    • /
    • pp.127-134
    • /
    • 2019
  • With the advancement of IT technology, the amount of data generated has been growing exponentially every year. As an alternative to this, research on distributed systems and in-memory based big data processing schemes has been actively underway. The processing power of traditional big data processing schemes enables big data to be processed as fast as the number of nodes and memory capacity increases. However, the increase in the number of nodes inevitably raises the frequency of failures in a big data infrastructure environment, and infrastructure management points and infrastructure operating costs also increase accordingly. In addition, the increase in memory capacity raises infrastructure costs for a node configuration. Therefore, this paper proposes an in-memory-based hybrid big data processing scheme for improve the big data processing rate. The proposed scheme reduces the number of nodes compared to traditional big data processing schemes based on distributed systems by adding a combiner step to a distributed system processing scheme and applying an in-memory based processing technology at that step. It decreases the big data processing time by approximately 22%. In the future, realistic performance evaluation in a big data infrastructure environment consisting of more nodes will be required for practical verification of the proposed scheme.

Big Data Platform for Utilizing and Analyzing Real-Time Sensing Information in Industrial Sites (산업현장 실시간 센싱정보 활용/분석을 위한 빅데이터 플랫폼)

  • Lee, Yonghwan;Suh, Jinhyung
    • Journal of Creative Information Culture
    • /
    • v.6 no.1
    • /
    • pp.15-21
    • /
    • 2020
  • In order to utilize big data in general industrial sites, the structured big data collected from facilities, processes, and environments of industrial sites must first be processed and stored, and in the case of unstructured data, it must be stored as unstructured data or converted into structured data and stored in a database. In this paper, we study a method of collecting big data based on open IoT standards that can converge and utilize measurement information, environmental information of industrial sites to collect big data. The platform for collecting big data proposed in this paper is capable of collecting, processing, and storing big data at industrial sites to process real-time sensing information. For processing and analyzing data according to the purpose of the stored industrial, various big data technologies also can be applied.