• Title/Summary/Keyword: in-memory data management

Search Result 431, Processing Time 0.031 seconds

Queue Memory Management Method for Continuous Query Processing in Data Stream (데이터 스트림에서 연속질의 처리를 위한 큐 메모리 관리 기법)

  • Shin, Jae-Wan;Shin, Soong-Sun;Lee, Dong-Wook;Kim, Kyung-Bae;Bae, Hae-Young
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2008.06c
    • /
    • pp.179-183
    • /
    • 2008
  • 연속적이고 무한히 발생되는 데이터 스트림을 관리하는 데이터 스트림 관리시스템(DSMS)은 연속질의를 이용하여 스트림을 처리한다. 연속질의는 질의 별로 독립적인 큐를 유지하기 때문에 질의 개수가 증가함에 따라서 메모리 비용이 증가되며, 잦은 메모리 할당으로 인한 시스템의 성능 저하를 갖는다. 이러한 문제점을 해결하기 위한 기존의 연구로 메모리 풀을 이용한 메모리 관리 기법이 있다. 하지만 페이지의 크기가 고정되어 있기 때문에 각 질의마다 필요로 하는 데이터 스트림의 최적의 크기에 적합하게 할당되지 못하여 메모리를 낭비하는 문제점이 있다. 본 논문에서는 이러한 문제를 해결하기 위해 연속질의 처리를 위한 큐 메모리 관리 기법을 제안한다. 제안기법은 큐 관리 테이블에서 관리하는 각각의 큐 메모리들을 타임스탬프를 가지고 일정한 기간을 주기로 큐 메모리의 사용량을 분석한다. 분석된 큐 메모리들은 이전의 큐 메모리의 사용량과 현재 사용된 큐 메모리의 사용량을 비교함으로써 상한 값과 하한 값을 구함으로써 현재 큐 메모리에서 가지고 있는 사용량을 추가할 것인지, 줄일 것인지를 판단하여, 메모리의 사용량을 최적화 함으로써 시스템의 메모리 가용성을 향상한다. 제안 기법은 성능평가를 통해 메모리의 가용성이 기존의 방식에 비하여 향상된 성능을 보인다.

  • PDF

A Study on the Recommendation Algorithm based on Trust/Distrust Relationship Network Analysis (사용자 간 신뢰·불신 관계 네트워크 분석 기반 추천 알고리즘에 관한 연구)

  • Noh, Heeryong;Ahn, Hyunchul
    • Journal of Information Technology Applications and Management
    • /
    • v.24 no.1
    • /
    • pp.169-185
    • /
    • 2017
  • This study proposes a novel recommendation algorithm that reflects the results from trust/distrust network analysis as a solution to enhance prediction accuracy of recommender systems. The recommendation algorithm of our study is based on memory-based collaborative filtering (CF), which is the most popular recommendation algorithm. But, unlike conventional CF, our proposed algorithm considers not only the correlation of the rating patterns between users, but also the results from trust/distrust relationship network analysis (e.g. who are the most trusted/distrusted users?, whom are the target user trust or distrust?) when calculating the similarity between users. To validate the performance of the proposed algorithm, we applied it to a real-world dataset that contained the trust/distrust relationships among users as well as their numeric ratings on movies. As a result, we found that the proposed algorithm outperformed the conventional CF with statistical significance. Also, we found that distrust relationship was more important than trust relationship in measuring similarities between users. This implies that we need to be more careful about negative relationship rather than positive one when tracking and managing social relationships among users.

Short-Term Water Quality Prediction of the Paldang Reservoir Using Recurrent Neural Network Models (순환신경망 모델을 활용한 팔당호의 단기 수질 예측)

  • Jiwoo Han;Yong-Chul Cho;Soyoung Lee;Sanghun Kim;Taegu Kang
    • Journal of Korean Society on Water Environment
    • /
    • v.39 no.1
    • /
    • pp.46-60
    • /
    • 2023
  • Climate change causes fluctuations in water quality in the aquatic environment, which can cause changes in water circulation patterns and severe adverse effects on aquatic ecosystems in the future. Therefore, research is needed to predict and respond to water quality changes caused by climate change in advance. In this study, we tried to predict the dissolved oxygen (DO), chlorophyll-a, and turbidity of the Paldang reservoir for about two weeks using long short-term memory (LSTM) and gated recurrent units (GRU), which are deep learning algorithms based on recurrent neural networks. The model was built based on real-time water quality data and meteorological data. The observation period was set from July to September in the summer of 2021 (Period 1) and from March to May in the spring of 2022 (Period 2). We tried to select an algorithm with optimal predictive power for each water quality parameter. In addition, to improve the predictive power of the model, an important variable extraction technique using random forest was used to select only the important variables as input variables. In both Periods 1 and 2, the predictive power after extracting important variables was further improved. Except for DO in Period 2, GRU was selected as the best model in all water quality parameters. This methodology can be useful for preventive water quality management by identifying the variability of water quality in advance and predicting water quality in a short period.

Behavioural Analysis of Password Authentication and Countermeasure to Phishing Attacks - from User Experience and HCI Perspectives (사용자의 패스워드 인증 행위 분석 및 피싱 공격시 대응방안 - 사용자 경험 및 HCI의 관점에서)

  • Ryu, Hong Ryeol;Hong, Moses;Kwon, Taekyoung
    • Journal of Internet Computing and Services
    • /
    • v.15 no.3
    • /
    • pp.79-90
    • /
    • 2014
  • User authentication based on ID and PW has been widely used. As the Internet has become a growing part of people' lives, input times of ID/PW have been increased for a variety of services. People have already learned enough to perform the authentication procedure and have entered ID/PW while ones are unconscious. This is referred to as the adaptive unconscious, a set of mental processes incoming information and producing judgements and behaviors without our conscious awareness and within a second. Most people have joined up for various websites with a small number of IDs/PWs, because they relied on their memory for managing IDs/PWs. Human memory decays with the passing of time and knowledges in human memory tend to interfere with each other. For that reason, there is the potential for people to enter an invalid ID/PW. Therefore, these characteristics above mentioned regarding of user authentication with ID/PW can lead to human vulnerabilities: people use a few PWs for various websites, manage IDs/PWs depending on their memory, and enter ID/PW unconsciously. Based on the vulnerability of human factors, a variety of information leakage attacks such as phishing and pharming attacks have been increasing exponentially. In the past, information leakage attacks exploited vulnerabilities of hardware, operating system, software and so on. However, most of current attacks tend to exploit the vulnerabilities of the human factors. These attacks based on the vulnerability of the human factor are called social-engineering attacks. Recently, malicious social-engineering technique such as phishing and pharming attacks is one of the biggest security problems. Phishing is an attack of attempting to obtain valuable information such as ID/PW and pharming is an attack intended to steal personal data by redirecting a website's traffic to a fraudulent copy of a legitimate website. Screens of fraudulent copies used for both phishing and pharming attacks are almost identical to those of legitimate websites, and even the pharming can include the deceptive URL address. Therefore, without the supports of prevention and detection techniques such as vaccines and reputation system, it is difficult for users to determine intuitively whether the site is the phishing and pharming sites or legitimate site. The previous researches in terms of phishing and pharming attacks have mainly studied on technical solutions. In this paper, we focus on human behaviour when users are confronted by phishing and pharming attacks without knowing them. We conducted an attack experiment in order to find out how many IDs/PWs are leaked from pharming and phishing attack. We firstly configured the experimental settings in the same condition of phishing and pharming attacks and build a phishing site for the experiment. We then recruited 64 voluntary participants and asked them to log in our experimental site. For each participant, we conducted a questionnaire survey with regard to the experiment. Through the attack experiment and survey, we observed whether their password are leaked out when logging in the experimental phishing site, and how many different passwords are leaked among the total number of passwords of each participant. Consequently, we found out that most participants unconsciously logged in the site and the ID/PW management dependent on human memory caused the leakage of multiple passwords. The user should actively utilize repudiation systems and the service provider with online site should support prevention techniques that the user can intuitively determined whether the site is phishing.

Flow rate prediction at Paldang Bridge using deep learning models (딥러닝 모형을 이용한 팔당대교 지점에서의 유량 예측)

  • Seong, Yeongjeong;Park, Kidoo;Jung, Younghun
    • Journal of Korea Water Resources Association
    • /
    • v.55 no.8
    • /
    • pp.565-575
    • /
    • 2022
  • Recently, in the field of water resource engineering, interest in predicting time series water levels and flow rates using deep learning technology that has rapidly developed along with the Fourth Industrial Revolution is increasing. In addition, although water-level and flow-rate prediction have been performed using the Long Short-Term Memory (LSTM) model and Gated Recurrent Unit (GRU) model that can predict time-series data, the accuracy of flow-rate prediction in rivers with rapid temporal fluctuations was predicted to be very low compared to that of water-level prediction. In this study, the Paldang Bridge Station of the Han River, which has a large flow-rate fluctuation and little influence from tidal waves in the estuary, was selected. In addition, time-series data with large flow fluctuations were selected to collect water-level and flow-rate data for 2 years and 7 months, which are relatively short in data length, to be used as training and prediction data for the LSTM and GRU models. When learning time-series water levels with very high time fluctuation in two models, the predicted water-level results in both models secured appropriate accuracy compared to observation water levels, but when training rapidly temporal fluctuation flow rates directly in two models, the predicted flow rates deteriorated significantly. Therefore, in this study, in order to accurately predict the rapidly changing flow rate, the water-level data predicted by the two models could be used as input data for the rating curve to significantly improve the prediction accuracy of the flow rates. Finally, the results of this study are expected to be sufficiently used as the data of flood warning system in urban rivers where the observation length of hydrological data is not relatively long and the flow-rate changes rapidly.

Priority-based Hint Management Scheme for Improving Page Sharing Opportunity of Virtual Machines (가상머신의 페이지 공유 기회를 향상시키기 위한 우선순위 큐 기반 힌트 관리 기법)

  • Nam, Yeji;Lee, Minho;Lee, Dongwoo;Eom, Young Ik
    • Journal of KIISE
    • /
    • v.43 no.9
    • /
    • pp.947-952
    • /
    • 2016
  • Most data centers attempt to consolidate servers using virtualization technology to efficiently utilize limited physical resources. Moreover, virtualized systems have commonly adopted contents-based page sharing mechanism for page deduplication among virtual machines (VMs). However, previous page sharing schemes are limited by the inability to effectively manage accumulated hints which mean sharable pages in stack. In this paper, we propose a priority-based hint management scheme to efficiently manage accumulated hints, which are sent from guest to host for improving page sharing opportunity in virtualized systems. Experimental results show that our scheme removes pages with low sharing potential, as compared with the previous schemes, by efficiently managing the accumulated pages.

Application of deep learning method for decision making support of dam release operation (댐 방류 의사결정지원을 위한 딥러닝 기법의 적용성 평가)

  • Jung, Sungho;Le, Xuan Hien;Kim, Yeonsu;Choi, Hyungu;Lee, Giha
    • Journal of Korea Water Resources Association
    • /
    • v.54 no.spc1
    • /
    • pp.1095-1105
    • /
    • 2021
  • The advancement of dam operation is further required due to the upcoming rainy season, typhoons, or torrential rains. Besides, physical models based on specific rules may sometimes have limitations in controlling the release discharge of dam due to inherent uncertainty and complex factors. This study aims to forecast the water level of the nearest station to the dam multi-timestep-ahead and evaluate the availability when it makes a decision for a release discharge of dam based on LSTM (Long Short-Term Memory) of deep learning. The LSTM model was trained and tested on eight data sets with a 1-hour temporal resolution, including primary data used in the dam operation and downstream water level station data about 13 years (2009~2021). The trained model forecasted the water level time series divided by the six lead times: 1, 3, 6, 9, 12, 18-hours, and compared and analyzed with the observed data. As a result, the prediction results of the 1-hour ahead exhibited the best performance for all cases with an average accuracy of MAE of 0.01m, RMSE of 0.015 m, and NSE of 0.99, respectively. In addition, as the lead time increases, the predictive performance of the model tends to decrease slightly. The model may similarly estimate and reliably predicts the temporal pattern of the observed water level. Thus, it is judged that the LSTM model could produce predictive data by extracting the characteristics of complex hydrological non-linear data and can be used to determine the amount of release discharge from the dam when simulating the operation of the dam.

UDP Flow Entry Management for Software-Defined Networking (사용자 정의 네트워크를 위한 사용자 데이터그램 프로토콜 플로우 엔트리 관리 기법)

  • Choi, Hanhimnara;Raza, Syed Muhammad;Kim, Moonseong;Choo, Hyunseung
    • Journal of Internet Computing and Services
    • /
    • v.22 no.2
    • /
    • pp.11-17
    • /
    • 2021
  • Software-defined networking provides a programmable and flexible way to manage the network by separating the control plane from data plane. However, the limited switch memory restricts the number of flow entries in the flow table used to forward packets. This leads to flow table overflow and flow entry reinstallation, which severely degrade the network performance. Therefore, this paper proposes a comprehensive policy for timely eviction of inactive flow entries to optimally maintain flow tables usage. In particular, statistics of user datagram protocol flow entries are periodically sampled to enable the inactive entries to be evicted early. Through traffic-based experiments, we found that the proposed system reduces the number of overflow occurrences and flow entries reinstallation compared to the random and FIFO policies.

A Study on the Thermal Prediction Model cf the Heat Storage Tank for the Optimal Use of Renewable Energy (신재생 에너지 최적 활용을 위한 축열조 온도 예측 모델 연구)

  • HanByeol Oh;KyeongMin Jang;JeeYoung Oh;MyeongBae Lee;JangWoo Park;YongYun Cho;ChangSun Shin
    • Smart Media Journal
    • /
    • v.12 no.10
    • /
    • pp.63-70
    • /
    • 2023
  • Recently, energy consumption for heating costs, which is 35% of smart farm energy costs, has increased, requiring energy consumption efficiency, and the importance of new and renewable energy is increasing due to concerns about the realization of electricity bills. Renewable energy belongs to hydropower, wind, and solar power, of which solar energy is a power generation technology that converts it into electrical energy, and this technology has less impact on the environment and is simple to maintain. In this study, based on the greenhouse heat storage tank and heat pump data, the factors that affect the heat storage tank are selected and a heat storage tank supply temperature prediction model is developed. It is predicted using Long Short-Term Memory (LSTM), which is effective for time series data analysis and prediction, and XGBoost model, which is superior to other ensemble learning techniques. By predicting the temperature of the heat pump heat storage tank, energy consumption may be optimized and system operation may be optimized. In addition, we intend to link it to the smart farm energy integrated operation system, such as reducing heating and cooling costs and improving the energy independence of farmers due to the use of solar power. By managing the supply of waste heat energy through the platform and deriving the maximum heating load and energy values required for crop growth by season and time, an optimal energy management plan is derived based on this.

A Study on QoS Measurement & Evaluation for MPEG Transmission in Network (통신망에서 MPEG 영상 전송을 위한 QoS 측정 및 평가에 관한 연구)

  • Suh Jae-Chul
    • Journal of Digital Contents Society
    • /
    • v.3 no.1
    • /
    • pp.101-111
    • /
    • 2002
  • Lately development of network around Internet expands range of data traffic to multimedia information, and so for the guarantee of multimedia services end-to-end QoS(Quality of Service) must service because comparing with existing Internet service can not support For satisfying those QoS requirements, network have to guarantee not only network on parameter, such as delay, jitter, throughput but also system resources like CPU utilization, memory usage. Therefore it is urgent to develop QoS based middleware to distribute multimedia data and maximize network utilization in the limited resource environment. And it must be necessary of network to provide end-to-end QoS(Quality of Service) for multimedia applications. Multimedia applications want that QoS which satisfy their own service properties be guaranteed Then, We must analyze those necessary QoS requirements md define QoS parameter which specify as two viewpoint, user's and network's perspective. Therefore network provider supplying network for usual user and university, enterprise must want to find about their own network performance and problem. It is essential for network manager to want to use a tool like this. On the basis of technique about QoS test-bed in the AIM network, We studied on the method of QoS measurement and management about end-to-end connection in the Internet. We measured network status about end-to-end connection and analyze the result of performance.

  • PDF