• Title/Summary/Keyword: Q learning

Search Result 428, Processing Time 0.037 seconds

A Learning-based Power Control Scheme for Edge-based eHealth IoT Systems

  • Su, Haoru;Yuan, Xiaoming;Tang, Yujie;Tian, Rui;Sun, Enchang;Yan, Hairong
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.12
    • /
    • pp.4385-4399
    • /
    • 2021
  • The Internet of Things (IoT) eHealth systems composed by Wireless Body Area Network (WBAN) has emerged recently. Sensor nodes are placed around or in the human body to collect physiological data. WBAN has many different applications, for instance health monitoring. Since the limitation of the size of the battery, besides speed, reliability, and accuracy; design of WBAN protocols should consider the energy efficiency and time delay. To solve these problems, this paper adopt the end-edge-cloud orchestrated network architecture and propose a transmission based on reinforcement algorithm. The priority of sensing data is classified according to certain application. System utility function is modeled according to the channel factors, the energy utility, and successful transmission conditions. The optimization problem is mapped to Q-learning model. Following this online power control protocol, the energy level of both the senor to coordinator, and coordinator to edge server can be modified according to the current channel condition. The network performance is evaluated by simulation. The results show that the proposed power control protocol has higher system energy efficiency, delivery ratio, and throughput.

The Design and Development of Online Learning Community for Teenagers (10대 학생들을 위한 온라인 학습 커뮤니티 설계 및 개발)

  • Jo, Mi-Heon
    • Journal of The Korean Association of Information Education
    • /
    • v.11 no.1
    • /
    • pp.79-89
    • /
    • 2007
  • Considering the continuous increase of teenagers' Internet use and the high rate of their participation in online communities, educators need to devote efforts to minimize the negative function of teenagers' Internet use and to guide them to experience positive aspects of Internet. Through activities in online communities, teenagers can experience various socialities and explore their self-consciousness. However, on the other hand many communities do not provide teenagers with considerate care. This may cause conflict in their sense of value and mislead them. Through this study, online learning community is designed and developed, which can reflect teenagers' various needs and connect their needs with learning experiences. The community consists of the following categories: 'about e-Tivity'(community vision, general information, list of activities, Q&A), 'activity space for teens'(event hall, data park, e-Tivity center), 'administrative space for teachers and managers'(research room, management room), and 'teen's private space'(my room, my avatar).

  • PDF

The Development and Effect Analysis of an Internet Based Nursing Program: Application to Nursing Informatics (인터넷을 이용한 간호학 교육 프로그램 개발 및 효과분석 -간호정보학을 중심으로-)

  • Yom, Young-Hee
    • Journal of Korean Academy of Nursing
    • /
    • v.30 no.4
    • /
    • pp.1035-1044
    • /
    • 2000
  • The purpose of this study was to develop and evaluate an internet based program for nursing informatics. The course subject, Nursing Informatics, was made by a computerized instructional module using the internet. The program was developed after taking into consideration the level of competence and knowledge in the subjects. It was based on 10 steps of the CAI module developed by Alessi and Trollip. The subjects consisted of 76 junior nursing students taking a Nursing Informatics course. Two sets of questionnaires were used for this study. First, a questionnaire was administered to 76 students to collect general information on their experience while using computers and the internet. Secondly, another questionnaire was administrated to 76 students after they took the course. They were asked to evaluate the program in terms of easiness of use, precision of contents, freshness of contents, motivation in learning, effectiveness of learning, enhancement of communication, precision of screen, and interest in the contents. IDs and passwords were given to the students. The students were asked to write their IDs and passwords when they connected to Nursing Informatics (http://hallym.ac.kr/~yhyom/ inform.html). They were led the menu page which was categorized into 8 icons (i. e., syllabus, lecture notes, quick test, Q & A board, assignment, on-line test, related web sites and mailing lists) after confirming their IDs and passwords. The students' responses were very positive. This program was a very useful in increasing the effectiveness of learning and motivation in the students. Suggest to be use for other nursing courses.

  • PDF

Advanced Control Techniques for Batch Processes Based on Iterative Learning Control Methods (반복학습제어를 기반으로 한 회분공정의 고급제어기법)

  • Lee, Kwang Soon
    • Korean Chemical Engineering Research
    • /
    • v.44 no.5
    • /
    • pp.425-434
    • /
    • 2006
  • The operability and productivity of continuous processes, especially in petrochemical industries have made remarkable improvement during the past twenty years through advanced process control (APC) typified by model-based predictive control. On the other hand, APC have not been actively practiced in industrial batch processes typified by batch polymerization reactors. Perhaps the main cause for this has been the lack of reliable batch process APC techniques that can overcome the unique problems in industrial batch processes. Recently, some noteworthy progress is being made in this area. New high-performance batch process control techniques that can accommodate and also overcome the unique problems of industrial batch processes have been proposed on the basis of iterative learning control (ILC). In this review paper, recent advancement in the batch process APC techniques are presented, with a particular focus on the variations of the so called Q-ILC method, with the hope that they are widely practiced in different industrial batch processes and enhance their operations.

Development of an Item Based Learning System Using E-mail (전자우편을 활용한 문항 기반 학습 시스템 개발)

  • Choi, Yong S.;Kim, Phil-Sun
    • The Journal of Korean Association of Computer Education
    • /
    • v.6 no.4
    • /
    • pp.85-93
    • /
    • 2003
  • This paper proposes an item based learning system using e-mail in order to motivate learners effectively and avoid the bottleneck problem under highly competitive access situations such as the traditional web based learning. The proposed system has th ree features as follows: first, through e-mail, a learner receives test items depending on his/her level, submit an answer sheet, and then identify an assessment with help messages. Secondly, on the web, an instructor easily constructs item database without any other tools and monitors the status of each learner by identifying the learners' record. Finally, an easy-to-use interaction mechanism enhances the inter-activity between learners and instructors, and the usability of Q&A(Question and Answer) service by incorporating e-mail into the web bulletin board.

  • PDF

Deep Learning-Based, Real-Time, False-Pick Filter for an Onsite Earthquake Early Warning (EEW) System (온사이트 지진조기경보를 위한 딥러닝 기반 실시간 오탐지 제거)

  • Seo, JeongBeom;Lee, JinKoo;Lee, Woodong;Lee, SeokTae;Lee, HoJun;Jeon, Inchan;Park, NamRyoul
    • Journal of the Earthquake Engineering Society of Korea
    • /
    • v.25 no.2
    • /
    • pp.71-81
    • /
    • 2021
  • This paper presents a real-time, false-pick filter based on deep learning to reduce false alarms of an onsite Earthquake Early Warning (EEW) system. Most onsite EEW systems use P-wave to predict S-wave. Therefore, it is essential to properly distinguish P-waves from noises or other seismic phases to avoid false alarms. To reduce false-picks causing false alarms, this study made the EEWNet Part 1 'False-Pick Filter' model based on Convolutional Neural Network (CNN). Specifically, it modified the Pick_FP (Lomax et al.) to generate input data such as the amplitude, velocity, and displacement of three components from 2 seconds ahead and 2 seconds after the P-wave arrival following one-second time steps. This model extracts log-mel power spectrum features from this input data, then classifies P-waves and others using these features. The dataset consisted of 3,189,583 samples: 81,394 samples from event data (727 events in the Korean Peninsula, 103 teleseismic events, and 1,734 events in Taiwan) and 3,108,189 samples from continuous data (recorded by seismic stations in South Korea for 27 months from 2018 to 2020). This model was trained with 1,826,357 samples through balancing, then tested on continuous data samples of the year 2019, filtering more than 99% of strong false-picks that could trigger false alarms. This model was developed as a module for USGS Earthworm and is written in C language to operate with minimal computing resources.

Design of weighted federated learning framework based on local model validation

  • Kim, Jung-Jun;Kang, Jeon Seong;Chung, Hyun-Joon;Park, Byung-Hoon
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.11
    • /
    • pp.13-18
    • /
    • 2022
  • In this paper, we proposed VW-FedAVG(Validation based Weighted FedAVG) which updates the global model by weighting according to performance verification from the models of each device participating in the training. The first method is designed to validate each local client model through validation dataset before updating the global model with a server side validation structure. The second is a client-side validation structure, which is designed in such a way that the validation data set is evenly distributed to each client and the global model is after validation. MNIST, CIFAR-10 is used, and the IID, Non-IID distribution for image classification obtained higher accuracy than previous studies.

A study on the improvement of ability of a creative solving mathematical problem (수학문제의 창의적 해결력 신장에 관한 연구 -농어촌 중학교 수학영재를 중심으로-)

  • 박형빈;서경식
    • Journal of the Korean School Mathematics Society
    • /
    • v.6 no.1
    • /
    • pp.1-17
    • /
    • 2003
  • In this paper, we study the methods of improving an ability of a creative solving mathematical problem belonging to an educational system which every province office of education has adopted for the mathematically talented students. Especially, we give an attention on a preferential reaction in teaching styles according to student's LQ., the relationship between student's LQ. and an ability of creative solving mathematical problems, and seeking for an appropriative teaching methods of the improvement ability of a creative solving problem. As results, we have the followings; 1. The group having excellent students who have a higher intelligential ability prefers inquiry learning which is composed of several sub-groups to a teacher-centered instruction. 2. The correlation coefficient between student's LQ. and an ability creative solving of mathematical is not high. 3. Although the contents and the model of thematic inquiry learning don't have a great influence on the divergent thinking (ex. fluency, flexibility, originality), they affect greatly the convergent thinking - a creative mathematical - problem solving ability. Accordingly, our results show that we should use a variety of mathematical teaching materials apart from our regular textbooks used in schools to improve a creative mathematical problem solving ability in the process of thematic inquiry learning. Also we can see that an inquiry learning which stimulates student's participation and discussion can be a desirable model in the thematic mathematical classroom activities.

  • PDF

SOM-Based State Generalization for Multiagent Reinforcement Learning (다중에이전트 강화학습을 위한 SOM기반의 상태 일한화)

  • 임문택;김인철
    • Proceedings of the Korea Inteligent Information System Society Conference
    • /
    • 2002.11a
    • /
    • pp.399-408
    • /
    • 2002
  • 다중 에이전트 학습이란 다중 에이전트 환경에서 에이전트간의 조정을 위한 행동전략을 학습하는 것을 말한다. 본 논문에서는 에이전트간의 통신이 불가능한 다중 에이전트 환경에서 각 에이전트들이 서로 독립적으로 대표적인 강화학습법인 Q학습을 전개함으로써 서로 효과적으로 협조할 수 있는 행동전략을 학습하려고 한다. 하지만 단일 에이전트 경우에 비해 보다 큰 상태-행동 공간을 갖는 다중 에이전트환경에서는 강화학습을 통해 효과적으로 최적의 행동 전략에 도달하기 어렵다는 문제점이 있다. 이 문제에 대한 기존의 접근방법은 크게 모듈화 방법과 일반화 방법이 제안되었으나 모두 나름의 제한을 가지고 있다. 본 논문에서는 대표적인 다중 에이전트 학습 문제의 예로서 먹이와 사냥꾼 문제(Prey and Hunters Problem)를 소개하고 이 문제영역을 통해 이와 같은 강화학습의 문제점을 살펴보고, 해결책으로 신경망 SOM을 이용한 일반화 방법인 QSOM 학습법을 제안한다. 이 방법은 기존의 일반화 방법과는 달리 군집화 기능을 제공하는 신경망 SOM을 이용함으로써 명확한 다수의 훈련 예가 없어도 효과적으로 이전에 경험하지 못했던 상태-행동들에 대한 Q값을 예측하고 이용할 수 있다는 장점이 있다. 또한 본 논문에서는 실험을 통해 QSOM 학습법의 일반화 효과와 성능을 평가하였다.

  • PDF

Development of Interior Self-driving Service Robot Using Embedded Board Based on Reinforcement Learning (강화학습 기반 임베디드 보드를 활용한 실내자율 주행 서비스 로봇 개발)

  • Oh, Hyeon-Tack;Baek, Ji-Hoon;Lee, Seung-Jin;Kim, Sang-Hoon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2018.10a
    • /
    • pp.537-540
    • /
    • 2018
  • 본 논문은 Jetson_TX2(임베디드 보드)의 ROS(Robot Operating System)기반으로 맵 지도를 작성하고, SLAM 및 DQN(Deep Q-Network)을 이용한 목적지까지의 이동명령(목표 선속도, 목표 각속도)을 자이로센서로 측정한 현재 각속도를 이용하여 Cortex-M3의 기반의 MCU(Micro Controllor Unit)에 하달하여 엔코더(encoder) 모터에서 측정한 현재 선속도와 자이로센서에서 측정한 각속도 값을 이용하여 PID제어를 통한 실내 자율주행 서비스 로봇.