• Title/Summary/Keyword: in-memory data grid

Search Result 58, Processing Time 0.024 seconds

Development of Information Technology Infrastructures through Construction of Big Data Platform for Road Driving Environment Analysis (도로 주행환경 분석을 위한 빅데이터 플랫폼 구축 정보기술 인프라 개발)

  • Jung, In-taek;Chong, Kyu-soo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.3
    • /
    • pp.669-678
    • /
    • 2018
  • This study developed information technology infrastructures for building a driving environment analysis platform using various big data, such as vehicle sensing data, public data, etc. First, a small platform server with a parallel structure for big data distribution processing was developed with H/W technology. Next, programs for big data collection/storage, processing/analysis, and information visualization were developed with S/W technology. The collection S/W was developed as a collection interface using Kafka, Flume, and Sqoop. The storage S/W was developed to be divided into a Hadoop distributed file system and Cassandra DB according to the utilization of data. Processing S/W was developed for spatial unit matching and time interval interpolation/aggregation of the collected data by applying the grid index method. An analysis S/W was developed as an analytical tool based on the Zeppelin notebook for the application and evaluation of a development algorithm. Finally, Information Visualization S/W was developed as a Web GIS engine program for providing various driving environment information and visualization. As a result of the performance evaluation, the number of executors, the optimal memory capacity, and number of cores for the development server were derived, and the computation performance was superior to that of the other cloud computing.

Explainable Photovoltaic Power Forecasting Scheme Using BiLSTM (BiLSTM 기반의 설명 가능한 태양광 발전량 예측 기법)

  • Park, Sungwoo;Jung, Seungmin;Moon, Jaeuk;Hwang, Eenjun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.11 no.8
    • /
    • pp.339-346
    • /
    • 2022
  • Recently, the resource depletion and climate change problem caused by the massive usage of fossil fuels for electric power generation has become a critical issue worldwide. According to this issue, interest in renewable energy resources that can replace fossil fuels is increasing. Especially, photovoltaic power has gaining much attention because there is no risk of resource exhaustion compared to other energy resources and there are low restrictions on installation of photovoltaic system. In order to use the power generated by the photovoltaic system efficiently, a more accurate photovoltaic power forecasting model is required. So far, even though many machine learning and deep learning-based photovoltaic power forecasting models have been proposed, they showed limited success in terms of interpretability. Deep learning-based forecasting models have the disadvantage of being difficult to explain how the forecasting results are derived. To solve this problem, many studies are being conducted on explainable artificial intelligence technique. The reliability of the model can be secured if it is possible to interpret how the model derives the results. Also, the model can be improved to increase the forecasting accuracy based on the analysis results. Therefore, in this paper, we propose an explainable photovoltaic power forecasting scheme based on BiLSTM (Bidirectional Long Short-Term Memory) and SHAP (SHapley Additive exPlanations).

Resource Scheduling Framework based on Resource Parameter Graph (자원인자 기반 스케줄링 프레임워크)

  • 배재환;권성호;김덕수;이강우
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.8 no.3
    • /
    • pp.19-31
    • /
    • 2003
  • For the implementation of large scale GRID systems, the performance scalability in resource scheduling is clearly to be addressed. In this research, we analyzed existing scheduling frameworks from the viewpoint of the performance and propose a novel resource scheduling framework called resource parameter based scheduling. Proposed scheduling framework consists of three components. The first is the resource parameter graph that expresses resource information via inter-resource relation and the composition base on the hierarchical structure. The second component is the resource parameter tree to be used for the implementation of the memory-based index of resource information. The third component is the resource information repository which mostly consists of static data to be used for the general resource information services. This paper presents the details of the framework.

  • PDF

OPENMP PARALLEL PERFORMANCE OF A CFD CODE ON MULTI-CORE SYSTEMS (멀티코어 시스템에서 쓰레드 수에 따른 CFD 코드의 OpenMP 병렬 성능)

  • Kim, J.K.;Jang, K.J.;Kim, T.Y.;Cho, D.R.;Kim, S.D.;Choi, J.Y.
    • Journal of computational fluids engineering
    • /
    • v.18 no.1
    • /
    • pp.83-90
    • /
    • 2013
  • OpenMP is becoming more and more useful as a simple parallel processing paradigm on SMP (Shared Memory Multi-Processors) computing environment with the development of multi-core processors. However, very few data is available publically regarding the OpenMP performance in CFD (Computational Fluid Dynamics). In the present study a CFD test suite is prepared for the performance evaluation of OpenMP on various multi-core systems. The test suite is composed of two-dimensional numerical simulations for inviscid/viscous and reacting/non-reacting flows using three different levels of grid systems. One to five test runs were carried out on various systems from dual-core dual threads to 16-core 32-threads systems by changing the number of threads engaged for each test up to 80. The results exhibit some interesting results and the lessons learned from the tests would be quite helpful for the further use of OpenMP for CFD studies using multi-core processor systems.

Prediction of Urban Flood Extent by LSTM Model and Logistic Regression (LSTM 모형과 로지스틱 회귀를 통한 도시 침수 범위의 예측)

  • Kim, Hyun Il;Han, Kun Yeun;Lee, Jae Yeong
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.40 no.3
    • /
    • pp.273-283
    • /
    • 2020
  • Because of climate change, the occurrence of localized and heavy rainfall is increasing. It is important to predict floods in urban areas that have suffered inundation in the past. For flood prediction, not only numerical analysis models but also machine learning-based models can be applied. The LSTM (Long Short-Term Memory) neural network used in this study is appropriate for sequence data, but it demands a lot of data. However, rainfall that causes flooding does not appear every year in a single urban basin, meaning it is difficult to collect enough data for deep learning. Therefore, in addition to the rainfall observed in the study area, the observed rainfall in another urban basin was applied in the predictive model. The LSTM neural network was used for predicting the total overflow, and the result of the SWMM (Storm Water Management Model) was applied as target data. The prediction of the inundation map was performed by using logistic regression; the independent variable was the total overflow and the dependent variable was the presence or absence of flooding in each grid. The dependent variable of logistic regression was collected through the simulation results of a two-dimensional flood model. The input data of the two-dimensional flood model were the overflow at each manhole calculated by the SWMM. According to the LSTM neural network parameters, the prediction results of total overflow were compared. Four predictive models were used in this study depending on the parameter of the LSTM. The average RMSE (Root Mean Square Error) for verification and testing was 1.4279 ㎥/s, 1.0079 ㎥/s for the four LSTM models. The minimum RMSE of the verification and testing was calculated as 1.1655 ㎥/s and 0.8797 ㎥/s. It was confirmed that the total overflow can be predicted similarly to the SWMM simulation results. The prediction of inundation extent was performed by linking the logistic regression with the results of the LSTM neural network, and the maximum area fitness was 97.33 % when more than 0.5 m depth was considered. The methodology presented in this study would be helpful in improving urban flood response based on deep learning methodology.

Development of 3D Reverse Time Migration Software for Ultra-high-resolution Seismic Survey (초고해상 탄성파 탐사를 위한 3차원 역시간 구조보정 프로그램 개발)

  • Kim, Dae-sik;Shin, Jungkyun;Ha, Jiho;Kang, Nyeon Keon;Oh, Ju-Won
    • Geophysics and Geophysical Exploration
    • /
    • v.25 no.3
    • /
    • pp.109-119
    • /
    • 2022
  • The computational efficiency of reverse time migration (RTM) based on numerical modeling is not secured due to the high-frequency band of several hundred Hz or higher for data acquired through a three-dimensional (3D) ultra-high-resolution (UHR) seismic survey. Therefore, this study develops an RTM program to derive high-quality 3D geological structures using UHR seismic data. In the traditional 3D RTM program, an excitation amplitude technique that stores only the maximum amplitude of the source wavefield and a domain-limiting technique that minimizes the modeling area where the source and receivers are located were used to significantly reduce memory usage and calculation time. The program developed through this study successfully derived a 3D migration image with a horizontal grid size of 1 m for the 3D UHR seismic survey data obtained from the Korea Institute of Geoscience and Mineral Resources in 2019, and geological analysis was conducted.

Coarse Grid Wave Hindcasting in the Yellow Sea Considering the Effect of Tide and Tidal Current (조석 및 조류 효과를 고려한 황해역 광역 파랑 수치모의 실험)

  • Chun, Hwusub;Ahn, Kyungmo
    • Journal of Korean Society of Coastal and Ocean Engineers
    • /
    • v.30 no.6
    • /
    • pp.286-297
    • /
    • 2018
  • In the present study, wave measurements at KOGA-W01 were analyzed and then the numerical wind waves simulations have been conducted to investigate the characteristics of wind waves in the Yellow sea. According to the present analysis, even though the location of the wave stations are close to the coastal region, the deep water waves are prevailed due to the short fetch length. Chun and Ahn's (2017a, b) numerical model has been extended to the Yellow Sea in this study. The effects of tide and tidal currents should be included in the model to accommodate the distinctive effect of large tidal range and tidal current in the Yellow Sea. The wave hindcasting results were compared with the wave measurements collected KOGA-W01 and Kyeockpo. The comparison shows the reasonable agreements between wave hindcastings and measured data, however the model significantly underestimate the wave period of swell waves from the south due to the narrow computational domain. Despite the poorly prediction in the significant wave period of swell waves which usually have small wave heights, the estimation of the extreme wave height and corresponding wave period shows good agreement with the measurement data.

Comparative Study of Commercial CFD Software Performance for Prediction of Reactor Internal Flow (원자로 내부유동 예측을 위한 상용 전산유체역학 소프트웨어 성능 비교 연구)

  • Lee, Gong Hee;Bang, Young Seok;Woo, Sweng Woong;Kim, Do Hyeong;Kang, Min Ku
    • Transactions of the Korean Society of Mechanical Engineers B
    • /
    • v.37 no.12
    • /
    • pp.1175-1183
    • /
    • 2013
  • Even if some CFD software developers and its users think that a state-of-the-art CFD software can be used to reasonably solve at least single-phase nuclear reactor safety problems, there remain limitations and uncertainties in the calculation result. From a regulatory perspective, the Korea Institute of Nuclear Safety (KINS) is presently conducting the performance assessment of commercial CFD software for nuclear reactor safety problems. In this study, to examine the prediction performance of commercial CFD software with the porous model in the analysis of the scale-down APR (Advanced Power Reactor Plus) internal flow, a simulation was conducted with the on-board numerical models in ANSYS CFX R.14 and FLUENT R.14. It was concluded that depending on the CFD software, the internal flow distribution of the scale-down APR was locally somewhat different. Although there was a limitation in estimating the prediction performance of the commercial CFD software owing to the limited amount of measured data, CFX R.14 showed more reasonable prediction results in comparison with FLUENT R.14. Meanwhile, owing to the difference in discretization methodology, FLUENT R.14 required more computational memory than CFX R.14 for the same grid system. Therefore, the CFD software suitable to the available computational resource should be selected for massively parallel computations.