• Title/Summary/Keyword: reward time

Search Result 169, Processing Time 0.023 seconds

Path selection algorithm for multi-path system based on deep Q learning (Deep Q 학습 기반의 다중경로 시스템 경로 선택 알고리즘)

  • Chung, Byung Chang;Park, Heasook
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.25 no.1
    • /
    • pp.50-55
    • /
    • 2021
  • Multi-path system is a system in which utilizes various networks simultaneously. It is expected that multi-path system can enhance communication speed, reliability, security of network. In this paper, we focus on path selection in multi-path system. To select optimal path, we propose deep reinforcement learning algorithm which is rewarded by the round-trip-time (RTT) of each networks. Unlike multi-armed bandit model, deep Q learning is applied to consider rapidly changing situations. Due to the delay of RTT data, we also suggest compensation algorithm of the delayed reward. Moreover, we implement testbed learning server to evaluate the performance of proposed algorithm. The learning server contains distributed database and tensorflow module to efficiently operate deep learning algorithm. By means of simulation, we showed that the proposed algorithm has better performance than lowest RTT about 20%.

PGA: An Efficient Adaptive Traffic Signal Timing Optimization Scheme Using Actor-Critic Reinforcement Learning Algorithm

  • Shen, Si;Shen, Guojiang;Shen, Yang;Liu, Duanyang;Yang, Xi;Kong, Xiangjie
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.11
    • /
    • pp.4268-4289
    • /
    • 2020
  • Advanced traffic signal timing method plays very important role in reducing road congestion and air pollution. Reinforcement learning is considered as superior approach to build traffic light timing scheme by many recent studies. It fulfills real adaptive control by the means of taking real-time traffic information as state, and adjusting traffic light scheme as action. However, existing works behave inefficient in complex intersections and they are lack of feasibility because most of them adopt traffic light scheme whose phase sequence is flexible. To address these issues, a novel adaptive traffic signal timing scheme is proposed. It's based on actor-critic reinforcement learning algorithm, and advanced techniques proximal policy optimization and generalized advantage estimation are integrated. In particular, a new kind of reward function and a simplified form of state representation are carefully defined, and they facilitate to improve the learning efficiency and reduce the computational complexity, respectively. Meanwhile, a fixed phase sequence signal scheme is derived, and constraint on the variations of successive phase durations is introduced, which enhances its feasibility and robustness in field applications. The proposed scheme is verified through field-data-based experiments in both medium and high traffic density scenarios. Simulation results exhibit remarkable improvement in traffic performance as well as the learning efficiency comparing with the existing reinforcement learning-based methods such as 3DQN and DDQN.

Development of Deep Learning Ensemble Modeling for Cryptocurrency Price Prediction : Deep 4-LSTM Ensemble Model (암호화폐 가격 예측을 위한 딥러닝 앙상블 모델링 : Deep 4-LSTM Ensemble Model)

  • Choi, Soo-bin;Shin, Dong-hoon;Yoon, Sang-Hyeak;Kim, Hee-Woong
    • Journal of Information Technology Services
    • /
    • v.19 no.6
    • /
    • pp.131-144
    • /
    • 2020
  • As the blockchain technology attracts attention, interest in cryptocurrency that is received as a reward is also increasing. Currently, investments and transactions are continuing with the expectation and increasing value of cryptocurrency. Accordingly, prediction for cryptocurrency price has been attempted through artificial intelligence technology and social sentiment analysis. The purpose of this paper is to develop a deep learning ensemble model for predicting the price fluctuations and one-day lag price of cryptocurrency based on the design science research method. This paper intends to perform predictive modeling on Ethereum among cryptocurrencies to make predictions more efficiently and accurately than existing models. Therefore, it collects data for five years related to Ethereum price and performs pre-processing through customized functions. In the model development stage, four LSTM models, which are efficient for time series data processing, are utilized to build an ensemble model with the optimal combination of hyperparameters found in the experimental process. Then, based on the performance evaluation scale, the superiority of the model is evaluated through comparison with other deep learning models. The results of this paper have a practical contribution that can be used as a model that shows high performance and predictive rate for cryptocurrency price prediction and price fluctuations. Besides, it shows academic contribution in that it improves the quality of research by following scientific design research procedures that solve scientific problems and create and evaluate new and innovative products in the field of information systems.

Differentially Expressed Genes in Period 2-Overexpressing Mice Striatum May Underlie Their Lower Sensitivity to Methamphetamine Addiction-Like Behavior

  • Sayson, Leandro Val;Kim, Mikyung;Jeon, Se Jin;Custodio, Raly James Perez;Lee, Hyun Jun;Ortiz, Darlene Mae;Cheong, Jae Hoon;Kim, Hee Jin
    • Biomolecules & Therapeutics
    • /
    • v.30 no.3
    • /
    • pp.238-245
    • /
    • 2022
  • Previous reports have demonstrated that genetic mechanisms greatly mediate responses to drugs of abuse, including methamphetamine (METH). The circadian gene Period 2 (Per2) has been previously associated with differential responses towards METH in mice. While the behavioral consequences of eliminating Per2 have been illustrated previously, Per2 overexpression has not yet been comprehensively described; although, Per2-overexpressing (Per2 OE) mice previously showed reduced sensitivity towards METH-induced addiction-like behaviors. To further elucidate this distinct behavior of Per2 OE mice to METH, we identified possible candidate biomarkers by determining striatal differentially expressed genes (DEGs) in both drug-naïve and METH-treated Per2 OE mice relative to wild-type (WT), through RNA sequencing. Of the several DEGs in drug naïve Per2 OE mice, we identified six genes that were altered after repeated METH treatment in WT mice, but not in Per2 OE mice. These results, validated by quantitative real-time polymerase chain reaction, could suggest that the identified DEGs might underlie the previously reported weaker METH-induced responses of Per2 OE mice compared to WT. Gene network analysis also revealed that Asic3, Hba-a1, and Rnf17 are possibly associated with Per2 through physical interactions and predicted correlations, and might potentially participate in addiction. Inhibiting the functional protein of Asic3 prior to METH administration resulted in the partial reduction of METH-induced conditioned place preference in WT mice, supporting a possible involvement of Asic3 in METH-induced reward. Although encouraging further investigations, our findings suggest that these DEGs, including Asic3, may play significant roles in the lower sensitivity of Per2 OE mice to METH.

A Study on the Recognition of Fire-fighters on Korean Civil Anti-Disaster Organization for Public Safety (국민안전을 위한 민간 방재조직에 대한 소방관들의 인식 연구)

  • Chae, Jong-Sik;Lee, Si-Young
    • Journal of Korea Entertainment Industry Association
    • /
    • v.15 no.2
    • /
    • pp.137-148
    • /
    • 2021
  • This study sought for an improvement plan of the overall preventive activity of Korean Citizen-Corps-Active-In-Disaster(CAIND) by utilizing the fact that The fire-fighters, who are focusing on social disaster response works, are surveyed for their awareness of the vivid expertise to Korean CAIND. This group is defined as the assistant organization in law about disaster prevention which, the local residents willingly made to handle the situation of disaster occurrence. Since the characteristics of volunteer activities are also inherent, related issues are also reviewed at the same time to resolve any unclear arguments by disaster prevention activity characteristics of Korean Citizen Corps Active in Disaster. Through the results, the study provided three major suggestions for an improvement plan. The results of the study are as follows: First, the quota management system of Korean CAIND that considers the characteristics of rural areas should be actively supplemented. Second, the current reward system for Korean CAIND activities at large disaster sites should be surely improved. Third, the current education and training system of Korean CAIND to satisfy regional conditions should be newly established. The results of this study are largely expected to be utilized as a basic data to develop Korean CAIND in the future.

Hakeem: An Arabic Application Aimed to Teaching Children First Aid using Augmented Reality

  • Al-ajlan, Monirah;Altukhays, Wujud;Alyousef, Deema;Almansour, Aljawharah;Alsukayt, Layan;Alajlan, Halah
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.5
    • /
    • pp.368-374
    • /
    • 2022
  • Children are by nature curious and enthusiastic about learning and love to explore and search for everything they see around them, but as a result of this exploration they may sometimes be exposed to dangerous situations ranging from falls to poisoning and suffocation. That is why when supporting a child's natural desire to explore the world and supporting his awareness of dangerous situations and good handling of them, helps him build a conscious scientific mind and enhance his curiosity in the natural world. It is not easy to imagine a difficult situation in which we or one of our family is in danger, unable to help ourselves or to help them in time, due to our complete ignorance of the rules of first aid. Hence the importance of learning first aid not only for the child but for the community and the world at large. "Hakeem" is an Arabic E-health educational application that aims to teach children from the age of six to eleven years first aid, in our belief that the seed of renaissance lies in the care and education of children, and the lack of Arabic content that aims to teach children first aid skills. The idea is to create a scenario in which the child is responsible for saving the person who will be in a dangerous situation using Augmented Reality (AR) technology, to increase engagement and interaction and provides a rich user experience, and according to the child's performance, he will get reward points. The game will have several levels: Beginner, Intermediate, and Hakeem, and based on the player's points he will get a title and move to the next level, and when he reaches the end, he will get the certificate.

Hybrid phishing site detection system with GRU-based shortened URL determination technique (GRU 기반 단축 URL 판별 기법을 적용한 하이브리드 피싱 사이트 탐지 시스템)

  • Hae-Soo Kim;Mi-Hui Kim
    • Journal of IKEEE
    • /
    • v.27 no.3
    • /
    • pp.213-219
    • /
    • 2023
  • According to statistics from the National Police Agency, smishing crimes using texts or messengers have increased dramatically since COVID-19. In addition, most of the cases of impersonation of public institutions reported to agency were related to vaccination and reward, and many methods were used to trick people into clicking on fake URLs (Uniform Resource Locators). When detecting them, URL-based detection methods cannot detect them properly if the information of the URL is hidden, and content-based detection methods are slow and use a lot of resources. In this paper, we propose a system for URL-based detection using transformer for regular URLs and content-based detection using XGBoost for shortened URLs through the process of determining shortened URLs using GRU(Gated Recurrent Units). The F1-Score of the proposed detection system was 94.86, and its average processing time was 5.4 seconds.

Optimal deployment of sonobuoy for unmanned aerial vehicles using reinforcement learning considering the target movement (표적의 이동을 고려한 강화학습 기반 무인항공기의 소노부이 최적 배치)

  • Geunyoung Bae;Juhwan Kang;Jungpyo Hong
    • The Journal of the Acoustical Society of Korea
    • /
    • v.43 no.2
    • /
    • pp.214-224
    • /
    • 2024
  • Sonobuoys are disposable devices that utilize sound waves for information gathering, detecting engine noises, and capturing various acoustic characteristics. They play a crucial role in accurately detecting underwater targets, making them effective detection systems in anti-submarine warfare. Existing sonobuoy deployment methods in multistatic systems often rely on fixed patterns or heuristic-based rules, lacking efficiency in terms of the number of sonobuoys deployed and operational time due to the unpredictable mobility of the underwater targets. Thus, this paper proposes an optimal sonobuoy placement strategy for Unmanned Aerial Vehicles (UAVs) to overcome the limitations of conventional sonobuoy deployment methods. The proposed approach utilizes reinforcement learning in a simulation-based experimental environment that considers the movements of the underwater targets. The Unity ML-Agents framework is employed, and the Proximal Policy Optimization (PPO) algorithm is utilized for UAV learning in a virtual operational environment with real-time interactions. The reward function is designed to consider the number of sonobuoys deployed and the cost associated with sound sources and receivers, enabling effective learning. The proposed reinforcement learning-based deployment strategy compared to the conventional sonobuoy deployment methods in the same experimental environment demonstrates superior performance in terms of detection success rate, deployed sonobuoy count, and operational time.

An Empirical Study of Elderly Volunteering Commitment and Their Life Satisfaction Based on Activity Theory (노인자원봉사자의 봉사활동 헌신과 생활만족에 관한 연구 - 활동이론을 중심으로 -)

  • Kim, Mee-Hye;Jung, Jin-Kyung
    • Korean Journal of Social Welfare
    • /
    • v.54
    • /
    • pp.221-243
    • /
    • 2003
  • Activity theory explains that the old people participate in social activity as a role-substitute from loss of roles given by work and family. It is possible to enhance their life satisfaction through these activities. Based on this activity theory, this study aims at explore the role substitute of voluntary activities and to analysis whether volunteering commitment has an effect on life satisfaction. Data were collected by using survey questionnaires to the elderly over 55 years old who participated in voluntary activity at 25 Volunteer Centers in Seoul. The Activity theory was operationalized by job or joblessness, family type, achievement type obtained through voluntary activity The results are as follows. (1) Job or joblessness has effect on the activity frequence and activity time. (2) Social achievement after voluntary activity has effect on the duration only (3) And the family type did not have any effect. These three variables of activity theory do not have an effect on life satisfaction. The elderly volunteering commitment was explained by variables other than activity theory such as reward, health, education, sex. And the elderly volunteer's life satisfaction were affected by the family types and their economic status. These results imply that the Korean elderly voluntary activities could be expained partially by Activity theory. Also for these elder volunteers's life satisfaction, qualitative respects such as achievements through voluntary activity, and concerns and support by agencies are more important than the time they imput in voluntary activities.

  • PDF

Customer-Centric CRM Implementation Case Study (고객중심의 CRM 구축비교 사례연구)

  • Lee, Ho-Seoub
    • Management & Information Systems Review
    • /
    • v.23
    • /
    • pp.25-40
    • /
    • 2007
  • In the highly competitive and divers world of financial market, customer is the single most important factor to company's survival. Especially, creating a relationship with valued customers is a key to success. CRM provides the mean to retain high value customers. It takes a prospect of what customers expect. Utilizing those knowledge can help the products and service meet the customers' needs, thereby maximizing customer satisfaction and company's profit. In this report, I am going to suggest a few ways to develop successful CRM in the life insurance industry. First, CRM should innovate the way of communication to keep pace with Web 2.0 era. In other words, the customer's needs should be caught by real-time communication than traditional off-line market research. Thus, the functionality and specification of products can be decided by customer's direct choice so that the customers are able to purchase the understanding and experience of the products. Second, CRM project should consider whether the initial strategy plan can promise the stable growth of customer at the first step. When planning strategy, the project needs to identify what customer wants and how to fulfill the needs with stable growth of the customer. In addition, the CRM should be developed by realizing that customer centric benefits ultimately guarantee the growth of the organization. Third, CRM systems should enhance the organization's ability to take the customer's insight in a 360 degree view and to capture the voice of the customer directly. In order to develop the best matched product package, more precise customer segmentation should be ahead of market segmentation strategy. Forth, the biggest reward from CRM will be a customer royalty program. Many successful banks are already planning and practicing customer royalty strategy. A comprehensive analysis of customers and their behavior allow organization to identify high value potential customers' needs and determine a strategy required to meet those needs. Even life insurance companies such as Prudential Korea are developing products designed for royal customers. Fifth, understanding and managing the experience of customer called Customer Experience Management also can increase customer satisfaction. Measuring only customers' experience and adapting it to marketing strategy make products position in the gap between the customers' expectation and experience not required by market. A key component of CEM is its application across all organizational functions. At last, the direction of change and development of CRM can be defined from the conceptualization of information technology represented by Ubiquitous and Web 2.0. Instead of just managing customer information, companies should take the initiative in personalized system with customer oriented strategy. Furthermore, with the regular communication between CRM stakeholders (Sales-Marketing-IT), customer's demand should be directly reflected to enterprise strategy in real time.

  • PDF