• Title/Summary/Keyword: Era of Big Data

Search Result 367, Processing Time 0.025 seconds

Performance Evaluation and Analysis of Multiple Scenarios of Big Data Stream Computing on Storm Platform

  • Sun, Dawei;Yan, Hongbin;Gao, Shang;Zhou, Zhangbing
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.7
    • /
    • pp.2977-2997
    • /
    • 2018
  • In big data era, fresh data grows rapidly every day. More than 30,000 gigabytes of data are created every second and the rate is accelerating. Many organizations rely heavily on real time streaming, while big data stream computing helps them spot opportunities and risks from real time big data. Storm, one of the most common online stream computing platforms, has been used for big data stream computing, with response time ranging from milliseconds to sub-seconds. The performance of Storm plays a crucial role in different application scenarios, however, few studies were conducted to evaluate the performance of Storm. In this paper, we investigate the performance of Storm under different application scenarios. Our experimental results show that throughput and latency of Storm are greatly affected by the number of instances of each vertex in task topology, and the number of available resources in data center. The fault-tolerant mechanism of Storm works well in most big data stream computing environments. As a result, it is suggested that a dynamic topology, an elastic scheduling framework, and a memory based fault-tolerant mechanism are necessary for providing high throughput and low latency services on Storm platform.

Efficient K-Anonymization Implementation with Apache Spark

  • Kim, Tae-Su;Kim, Jong Wook
    • Journal of the Korea Society of Computer and Information
    • /
    • v.23 no.11
    • /
    • pp.17-24
    • /
    • 2018
  • Today, we are living in the era of data and information. With the advent of Internet of Things (IoT), the popularity of social networking sites, and the development of mobile devices, a large amount of data is being produced in diverse areas. The collection of such data generated in various area is called big data. As the importance of big data grows, there has been a growing need to share big data containing information regarding an individual entity. As big data contains sensitive information about individuals, directly releasing it for public use may violate existing privacy requirements. Thus, privacy-preserving data publishing (PPDP) has been actively studied to share big data containing personal information for public use, while preserving the privacy of the individual. K-anonymity, which is the most popular method in the area of PPDP, transforms each record in a table such that at least k records have the same values for the given quasi-identifier attributes, and thus each record is indistinguishable from other records in the same class. As the size of big data continuously getting larger, there is a growing demand for the method which can efficiently anonymize vast amount of dta. Thus, in this paper, we develop an efficient k-anonymity method by using Spark distributed framework. Experimental results show that, through the developed method, significant gains in processing time can be achieved.

Big Data, Business Analytics, and IoT: The Opportunities and Challenges for Business (빅데이터, 비즈니스 애널리틱스, IoT: 경영의 새로운 도전과 기회)

  • Jang, Young Jae
    • The Journal of Information Systems
    • /
    • v.24 no.4
    • /
    • pp.139-152
    • /
    • 2015
  • With the advancement of the Internet/IT technologies and the increased computation power, massive data can be collected, stored, and processed these days. The availability of large databases has brought forth a new era in which companies are hard pressed to find innovative ways to utilize immense amounts of data at their disposal. Indeed, data has opened a new age of business operations and management. There are already many cases of innovative businesses reaping success thanks to scientific decisions based on data analysis and mathematical algorithms. Big Data is a new paradigm in itself. In this article, Big Data is viewed as a new perspective rather than a new technology. This value centric definition of Big Data provides a new insight and opportunities. Moreover, the Business Analytics, which is the framework of creating tangible results in management, is introduced. Then the Internet of Things (IoT), another innovative concept of data collection and networking, is presented and how this new concept can be interpreted with Big Data in terms of the value centric perspective. The challenges and opportunities with these new concepts are also discussed.

Analysis of Factors Affecting Satisfaction with Commuting Time in the Era of Autonomous Driving (자율주행시대에 통근시간 만족도에 영향을 미치는 요인분석)

  • Jang, Jae-min;Cheon, Seung-hoon;Lee, Soong-bong
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.20 no.5
    • /
    • pp.172-185
    • /
    • 2021
  • As the era of autonomous driving approaches, it is expected to have a significant impact on our lives. When autonomous driving cars emerge, it is necessary to develop an index that can evaluate autonomous driving cars as it enhance the productive value of the car by reducing the burden on the driver. This study analyzed how the autonomous driving era affects commuting time and commuting time satisfaction among office goers using a car in Gyeonggi-do. First, a nonlinear relationship (V) was derived for the commuting time and commuting time satisfaction. Here, the factors affecting commuting time satisfaction were analyzed through a binomial logistic model, centered on the sample belonging to the nonlinear section (70 minutes or more for commuting time), which is likely to be affected by the autonomous driving era. The analysis results show that the variables affected by the autonomous driving era were health, sleeping hours, working hours, and leisure time. Since the emergence of autonomous driving cars is highly likely to improve the influencing variables, long-distance commuters are likely to feel higher commuting time satisfaction.

A Case Study on the Distribution of Cultural Contents in the Untact Era Using Big Data (빅데이터를 활용한 언택트 시대의 1인 콘텐츠 유통 사례 분석)

  • Wang, Deok-won;Kim, Jeong-hyeon;Son, Hye-ji;Jeon, Min-jun;Choi, Hun
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2021.05a
    • /
    • pp.301-302
    • /
    • 2021
  • After the Korona 19, "social distancing" was implemented, existing "pop culture" or entertainment programs were unable to communicate in both directions and declined. Since then, "Untact content" has shown its potential to grow due to untouch performances such as BTS' "Bangbangcon" and the rapid growth of Netflix, a global OTT (online video service). In addition, most of the global and Untact content is online and digital, which means a huge amount of big data will be poured out. Therefore, analyzing the big data poured out during the distribution of untact content will help us identify consumers' needs, and the growth expectations will also be high. Therefore, we would like to explore the research cases that have been conducted in existing studies regarding the subject of the study and analyze how big data can affect the distribution of content in the Untact era.

  • PDF

Utilization of Social Media Analysis using Big Data (빅 데이터를 이용한 소셜 미디어 분석 기법의 활용)

  • Lee, Byoung-Yup;Lim, Jong-Tae;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.13 no.2
    • /
    • pp.211-219
    • /
    • 2013
  • The analysis method using Big Data has evolved based on the Big data Management Technology. There are quite a few researching institutions anticipating new era in data analysis using Big Data and IT vendors has been sided with them launching standardized technologies for Big Data management technologies. Big Data is also affected by improvements of IT gadgets IT environment. Foreran by social media, analyzing method of unstructured data is being developed focusing on diversity of analyzing method, anticipation and optimization. In the past, data analyzing methods were confined to the optimization of structured data through data mining, OLAP, statics analysis. This data analysis was solely used for decision making for Chief Officers. In the new era of data analysis, however, are evolutions in various aspects of technologies; the diversity in analyzing method using new paradigm and the new data analysis experts and so forth. In addition, new patterns of data analysis will be found with the development of high performance computing environment and Big Data management techniques. Accordingly, this paper is dedicated to define the possible analyzing method of social media using Big Data. this paper is proposed practical use analysis for social media analysis through data mining analysis methodology.

Big Data Activation Plan for Digital Transformation of Agriculture and Rural (농업·농촌 디지털 전환을 위한 빅데이터 활성화 방안 연구)

  • Lee, Won Suk;Son, Kyungja;Jun, Daeho;Shin, Yongtae
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.9 no.8
    • /
    • pp.235-242
    • /
    • 2020
  • In order to promote digital transformation of our agricultural and rural communities in the wake of the fourth industrial revolution and prepare for the upcoming artificial intelligence era, it is necessary to establish a system and system that can collect, analyze and utilize necessary quality data. To this end, we will investigate and analyze problems and issues felt by various stakeholders such as farmers and agricultural officials, and present strategic measures to revitalize big data, which must be decided in order to promote digital transformation of our agricultural and rural communities, such as expanding big data platforms for joint utilization, establishing sustainable big data governance, and revitalizing the foundation for big data utilization based on demand.

Analysis of the Status of Artificial Medical Intelligence Technology Based on Big Data

  • KIM, Kyung-A;CHUNG, Myung-Ae
    • Korean Journal of Artificial Intelligence
    • /
    • v.10 no.2
    • /
    • pp.13-18
    • /
    • 2022
  • The role of artificial medical intelligence through medical big data has been focused on data-based medical device business and medical service technology development in the field of diagnostic examination of the patient's current condition, clinical decision support, and patient monitoring and management. Recently, with the 4th Industrial Revolution, the medical field changed the medical treatment paradigm from the method of treatment based on the knowledge and experience of doctors in the past to the form of receiving the help of high-precision medical intelligence based on medical data. In addition, due to the spread of non-face-to-face treatment due to the COVID-19 pandemic, it is expected that the era of telemedicine, in which patients will be treated by doctors at home rather than hospitals, will soon come. It can be said that artificial medical intelligence plays a big role at the center of this paradigm shift in prevention-centered treatment rather than treatment. Based on big data, this paper analyzes the current status of artificial intelligence technology for chronic disease patients, market trends, and domestic and foreign company trends to predict the expected effect and future development direction of artificial intelligence technology for chronic disease patients. In addition, it is intended to present the necessity of developing digital therapeutics that can provide various medical services to chronically ill patients and serve as medical support to clinicians.

A study on the success factors of Big Data through an analysis of introduction effect of Big Data (빅데이터 도입 효과 분석을 통한 빅데이터 성공요인에 관한 연구)

  • Jung, Young-Ki;Suk, Myung-Gun;Kim, Chang-Jae
    • Journal of Digital Convergence
    • /
    • v.12 no.11
    • /
    • pp.241-248
    • /
    • 2014
  • It has been expanded the bandwidth of data usages due to the rapid developments of information technology and infra hardware and then it was proposed to new paradigm of Big Data era. It has a trend to increase a Big Data technology and its performance gradually, thus enterprises have realized the importance of Data and the movement to take advantage of Big Data becomes active. This study has been performed to verify the importance through select the factors in order to active adoption of Big Data technology and utilization when enterprises use Big Data. It was selected that Big Data characteristic factors are the natures of predictability, manageability, affordability, competitiveness, creativity, responsiveness and supportability on the study. It is verified and showed that manageability were influenced to introduce Big Data in order, at the result of survey and statistics for enterprise practitioners who have big data experience.

Big Numeric Data Classification Using Grid-based Bayesian Inference in the MapReduce Framework

  • Kim, Young Joon;Lee, Keon Myung
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.14 no.4
    • /
    • pp.313-321
    • /
    • 2014
  • In the current era of data-intensive services, the handling of big data is a crucial issue that affects almost every discipline and industry. In this study, we propose a classification method for large volumes of numeric data, which is implemented in a distributed programming framework, i.e., MapReduce. The proposed method partitions the data space into a grid structure and it then models the probability distributions of classes for grid cells by collecting sufficient statistics using distributed MapReduce tasks. The class labeling of new data is achieved by k-nearest neighbor classification based on Bayesian inference.