• Title/Summary/Keyword: Big-Data Platform

Search Result 506, Processing Time 0.026 seconds

Big Data Astronomy: Large-scale Graph Analyses of Five Different Multiverses

  • Hong, Sungryong
    • The Bulletin of The Korean Astronomical Society
    • /
    • v.43 no.2
    • /
    • pp.36.3-37
    • /
    • 2018
  • By utilizing large-scale graph analytic tools in the modern Big Data platform, Apache Spark, we investigate the topological structures of five different multiverses produced by cosmological n-body simulations with various cosmological initial conditions: (1) one standard universe, (2) two different dark energy states, and (3) two different dark matter densities. For the Big Data calculations, we use a custom build of stand-alone Spark cluster at KIAS and Dataproc Compute Engine in Google Cloud Platform with the sample sizes ranging from 7 millions to 200 millions. Among many graph statistics, we find that three simple graph measurements, denoted by (1) $n_\k$, (2) $\tau_\Delta$, and (3) $n_{S\ge5}$, can efficiently discern different topology in discrete point distributions. We denote this set of three graph diagnostics by kT5+. These kT5+ statistics provide a quick look of various orders of n-points correlation functions in a computationally cheap way: (1) $n = 2$ by $n_k$, (2) $n = 3$ by $\tau_\Delta$, and (3) $n \ge 5$ by $n_{S\ge5}$.

  • PDF

Service Platform of Regional Smart Tour Ecosystem Support (지역중심의 스마트관광 생태계 지원 서비스 플랫)

  • Weon, Dalsoo
    • The Journal of the Convergence on Culture Technology
    • /
    • v.4 no.4
    • /
    • pp.31-36
    • /
    • 2018
  • The tourism industry has a great influence on national economy activation. The development of IT technology has enabled the collection and analysis of personal profile information, location information and activity information based on the characteristics, behavior, purchase propensity and interest of tourists. In order to realize this, the implementation of convergence smart tourism information service platform is completed by developing business model, IoT & Big Data integration management system, big data algorithm development and analysis platform in three stages. The underlying technology of the platform and algorithm needs a process of adopting open source, expanding the service element on the basis of it, and then complementing the problem through the test-bed demonstration test that connects the area. Using this platform, it is possible to develop a smart tourism environment that can provide customized services for each tourist by analyzing various information in an integrated manner. Also, it will be possible to improve the life of tourist destination residents and contribute to regional revitalization and job creation through the creation of smart tourism ecosystem focused on the region.

Analysis of Traffic Card Big Data by Hadoop and Sequential Mining Technique (하둡과 순차패턴 마이닝 기술을 통한 교통카드 빅데이터 분석)

  • Kim, Woosaeng;Kim, Yong Hoon;Park, Hee-Sung;Park, Jin-Kyu
    • Journal of Information Technology Applications and Management
    • /
    • v.24 no.4
    • /
    • pp.187-196
    • /
    • 2017
  • It is urgent to prepare countermeasures for traffic congestion problems of Korea's metropolitan area where central functions such as economic, social, cultural, and education are excessively concentrated. Most users of public transportation in metropolitan areas including Seoul use the traffic cards. If various information is extracted from traffic big data produced by the traffic cards, they can provide basic data for transport policies, land usages, or facility plans. Therefore, in this study, we extract valuable information such as the subway passengers' frequent travel patterns from the big traffic data provided by the Seoul Metropolitan Government Big Data Campus. For this, we use a Hadoop (High-Availability Distributed Object-Oriented Platform) to preprocess the big data and store it into a Mongo database in order to analyze it by a sequential pattern data mining technique. Since we analysis the actual big data, that is, the traffic cards' data provided by the Seoul Metropolitan Government Big Data Campus, the analyzed results can be used as an important referenced data when the Seoul government makes a plan about the metropolitan traffic policies.

A Context-Awareness Modeling User Profile Construction Method for Personalized Information Retrieval System

  • Kim, Jee Hyun;Gao, Qian;Cho, Young Im
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.14 no.2
    • /
    • pp.122-129
    • /
    • 2014
  • Effective information gathering and retrieval of the most relevant web documents on the topic of interest is difficult due to the large amount of information that exists in various formats. Current information gathering and retrieval techniques are unable to exploit semantic knowledge within documents in the "big data" environment; therefore, they cannot provide precise answers to specific questions. Existing commercial big data analytic platforms are restricted to a single data type; moreover, different big data analytic platforms are effective at processing different data types. Therefore, the development of a common big data platform that is suitable for efficiently processing various data types is needed. Furthermore, users often possess more than one intelligent device. It is therefore important to find an efficient preference profile construction approach to record the user context and personalized applications. In this way, user needs can be tailored according to the user's dynamic interests by tracking all devices owned by the user.

Study on Data Processing of the IOT Sensor Network Based on a Hadoop Cloud Platform and a TWLGA Scheduling Algorithm

  • Li, Guoyu;Yang, Kang
    • Journal of Information Processing Systems
    • /
    • v.17 no.6
    • /
    • pp.1035-1043
    • /
    • 2021
  • An Internet of Things (IOT) sensor network is an effective solution for monitoring environmental conditions. However, IOT sensor networks generate massive data such that the abilities of massive data storage, processing, and query become technical challenges. To solve the problem, a Hadoop cloud platform is proposed. Using the time and workload genetic algorithm (TWLGA), the data processing platform enables the work of one node to be shared with other nodes, which not only raises efficiency of one single node but also provides the compatibility support to reduce the possible risk of software and hardware. In this experiment, a Hadoop cluster platform with TWLGA scheduling algorithm is developed, and the performance of the platform is tested. The results show that the Hadoop cloud platform is suitable for big data processing requirements of IOT sensor networks.

A Study on the Accounts Balancing Time of Small Distributed Power Trading Platform Using Block Chain Network (블록체인 네트워크를 이용한 소규모 분산전력 거래플랫폼의 정산소요시간에 관한 연구)

  • Kim, Young-Gon;Heo, Keol;Choi, Jung-In;Wie, Jae-Woo
    • Journal of Energy Engineering
    • /
    • v.27 no.4
    • /
    • pp.86-91
    • /
    • 2018
  • This paper is a review of accounts balancing time in small distributed power trading platform using blockchain technology. First, the national VPP energy management system using the AMI applied to this study is introduced and then the accounts balancing time and process of the cryptocurrency coin payment which based on the power generation of pro-consumer certified by power big data analysis in a test bed environment is discussed. Futhermore the configuration of a power Big Data analysis system with GPU Fast Big Data that applies MapD to current lambda architecture is also introduced.

Estimation of Material Requirement of Piping Materials in an Offshore Structure using Big Data Analysis (빅데이터 분석을 이용한 해양 구조물 배관 자재의 소요량 예측)

  • Oh, Min-Jae;Roh, Myung-Il;Park, Sung-Woo;Kim, Seong-Hoon
    • Journal of the Society of Naval Architects of Korea
    • /
    • v.55 no.3
    • /
    • pp.243-251
    • /
    • 2018
  • In the shipyard, a lot of data is generated, stored, and managed during design, construction, and operation phases to build ships and offshore structures. However, it is difficult to handle such big data efficiently using existing data-handling technologies. As the big data technology is developed, the ship and offshore industries start to focus on the existing big data to find valuable information from it. In this paper, the material requirement estimation method of offshore structure piping materials using big data analysis is proposed. A big data platform for the data analysis in the shipyard is introduced and it is applied to the analysis of material requirement estimation to solve the problems in piping design by a designer. The regression model is developed from the big data of piping materials and verified using the existing data. This analysis can help a piping designer to estimate the exact amount of material requirement and schedule the purchase time.

Analysis of Success Factors of OTT Original Contents Through BigData, Netflix's 'Squid Game Season 2' Proposal (빅데이터를 통한 OTT 오리지널 콘텐츠의 성공요인 분석, 넷플릭스의 '오징어게임 시즌2' 제언)

  • Ahn, Sunghun;Jung, JaeWoo;Oh, Sejong
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.18 no.1
    • /
    • pp.55-64
    • /
    • 2022
  • This study analyzes the success factors of OTT original content through big data, and intends to suggest scenarios, casting, fun, and moving elements when producing the next work. In addition, I would like to offer suggestions for the success of 'Squid Game Season 2'. The success factor of 'Squid Game' through big data is first, it is a simple psychological experimental game. Second, it is a retro strategy. Third, modern visual beauty and color. Fourth, it is simple aesthetics. Fifth, it is the platform of OTT Netflix. Sixth, Netflix's video recommendation algorithm. Seventh, it induced Binge-Watch. Lastly, it can be said that the consensus was high as it was related to the time to think about 'death' and 'money' in a pandemic situation. The suggestions for 'Squid Game Season 2' are as follows. First, it is a fusion of famous traditional games of each country. Second, it is an AI-based planned MD product production and sales strategy. Third, it is casting based on artificial intelligence big data. Fourth, secondary copyright and copyright sales strategy. The limitations of this study were analyzed only through external data. Data inside the Netflix platform was not utilized. In this study, if AI big data is used not only in the OTT field but also in entertainment and film companies, it will be possible to discover better business models and generate stable profits.

A Study on Deep Learning Model-based Object Classification for Big Data Environment

  • Kim, Jeong-Sig;Kim, Jinhong
    • Journal of Software Assessment and Valuation
    • /
    • v.17 no.1
    • /
    • pp.59-66
    • /
    • 2021
  • Recently, conceptual information model is changing fast, and these changes are coming about as a result of individual tendency, social cultural, new circumstances and societal shifts within big data environment. Despite the data is growing more and more, now is the time to commit ourselves to the development of renewable, invaluable information of social/live commerce. Because we have problems with various insoluble data, we propose about deep learning prediction model-based object classification in social commerce of big data environment. Accordingly, it is an increased need of social commerce platform capable of handling high volumes of multiple items by users. Consequently, responding to rapid changes in users is a very significant by deep learning. Namely, promptly meet the needs of the times, and a widespread growth in big data environment with the goal of realizing in this paper.

Renewable energy trends and relationship structure by SNS big data analysis (SNS 빅데이터 분석을 통한 재생에너지 동향 및 관계구조)

  • Jong-Min Kim
    • Convergence Security Journal
    • /
    • v.22 no.1
    • /
    • pp.55-60
    • /
    • 2022
  • This study is to analyze trends and relational structures in the energy sector related to renewable energy. For this reason, in this study, we focused on big data including SNS data. SNS utilizes the Instagram platform to collect renewable energy hash tags and use them as a word embedding method for big data analysis and social network analysis, and based on the results derived from this research, it will be used for the development of the renewable energy industry. It is expected that it can be utilized.