• Title/Summary/Keyword: Q learning

Search Result 422, Processing Time 0.048 seconds

Deep Learning-Based, Real-Time, False-Pick Filter for an Onsite Earthquake Early Warning (EEW) System (온사이트 지진조기경보를 위한 딥러닝 기반 실시간 오탐지 제거)

  • Seo, JeongBeom;Lee, JinKoo;Lee, Woodong;Lee, SeokTae;Lee, HoJun;Jeon, Inchan;Park, NamRyoul
    • Journal of the Earthquake Engineering Society of Korea
    • /
    • v.25 no.2
    • /
    • pp.71-81
    • /
    • 2021
  • This paper presents a real-time, false-pick filter based on deep learning to reduce false alarms of an onsite Earthquake Early Warning (EEW) system. Most onsite EEW systems use P-wave to predict S-wave. Therefore, it is essential to properly distinguish P-waves from noises or other seismic phases to avoid false alarms. To reduce false-picks causing false alarms, this study made the EEWNet Part 1 'False-Pick Filter' model based on Convolutional Neural Network (CNN). Specifically, it modified the Pick_FP (Lomax et al.) to generate input data such as the amplitude, velocity, and displacement of three components from 2 seconds ahead and 2 seconds after the P-wave arrival following one-second time steps. This model extracts log-mel power spectrum features from this input data, then classifies P-waves and others using these features. The dataset consisted of 3,189,583 samples: 81,394 samples from event data (727 events in the Korean Peninsula, 103 teleseismic events, and 1,734 events in Taiwan) and 3,108,189 samples from continuous data (recorded by seismic stations in South Korea for 27 months from 2018 to 2020). This model was trained with 1,826,357 samples through balancing, then tested on continuous data samples of the year 2019, filtering more than 99% of strong false-picks that could trigger false alarms. This model was developed as a module for USGS Earthworm and is written in C language to operate with minimal computing resources.

Design of weighted federated learning framework based on local model validation

  • Kim, Jung-Jun;Kang, Jeon Seong;Chung, Hyun-Joon;Park, Byung-Hoon
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.11
    • /
    • pp.13-18
    • /
    • 2022
  • In this paper, we proposed VW-FedAVG(Validation based Weighted FedAVG) which updates the global model by weighting according to performance verification from the models of each device participating in the training. The first method is designed to validate each local client model through validation dataset before updating the global model with a server side validation structure. The second is a client-side validation structure, which is designed in such a way that the validation data set is evenly distributed to each client and the global model is after validation. MNIST, CIFAR-10 is used, and the IID, Non-IID distribution for image classification obtained higher accuracy than previous studies.

A study on the improvement of ability of a creative solving mathematical problem (수학문제의 창의적 해결력 신장에 관한 연구 -농어촌 중학교 수학영재를 중심으로-)

  • 박형빈;서경식
    • Journal of the Korean School Mathematics Society
    • /
    • v.6 no.1
    • /
    • pp.1-17
    • /
    • 2003
  • In this paper, we study the methods of improving an ability of a creative solving mathematical problem belonging to an educational system which every province office of education has adopted for the mathematically talented students. Especially, we give an attention on a preferential reaction in teaching styles according to student's LQ., the relationship between student's LQ. and an ability of creative solving mathematical problems, and seeking for an appropriative teaching methods of the improvement ability of a creative solving problem. As results, we have the followings; 1. The group having excellent students who have a higher intelligential ability prefers inquiry learning which is composed of several sub-groups to a teacher-centered instruction. 2. The correlation coefficient between student's LQ. and an ability creative solving of mathematical is not high. 3. Although the contents and the model of thematic inquiry learning don't have a great influence on the divergent thinking (ex. fluency, flexibility, originality), they affect greatly the convergent thinking - a creative mathematical - problem solving ability. Accordingly, our results show that we should use a variety of mathematical teaching materials apart from our regular textbooks used in schools to improve a creative mathematical problem solving ability in the process of thematic inquiry learning. Also we can see that an inquiry learning which stimulates student's participation and discussion can be a desirable model in the thematic mathematical classroom activities.

  • PDF

SOM-Based State Generalization for Multiagent Reinforcement Learning (다중에이전트 강화학습을 위한 SOM기반의 상태 일한화)

  • 임문택;김인철
    • Proceedings of the Korea Inteligent Information System Society Conference
    • /
    • 2002.11a
    • /
    • pp.399-408
    • /
    • 2002
  • 다중 에이전트 학습이란 다중 에이전트 환경에서 에이전트간의 조정을 위한 행동전략을 학습하는 것을 말한다. 본 논문에서는 에이전트간의 통신이 불가능한 다중 에이전트 환경에서 각 에이전트들이 서로 독립적으로 대표적인 강화학습법인 Q학습을 전개함으로써 서로 효과적으로 협조할 수 있는 행동전략을 학습하려고 한다. 하지만 단일 에이전트 경우에 비해 보다 큰 상태-행동 공간을 갖는 다중 에이전트환경에서는 강화학습을 통해 효과적으로 최적의 행동 전략에 도달하기 어렵다는 문제점이 있다. 이 문제에 대한 기존의 접근방법은 크게 모듈화 방법과 일반화 방법이 제안되었으나 모두 나름의 제한을 가지고 있다. 본 논문에서는 대표적인 다중 에이전트 학습 문제의 예로서 먹이와 사냥꾼 문제(Prey and Hunters Problem)를 소개하고 이 문제영역을 통해 이와 같은 강화학습의 문제점을 살펴보고, 해결책으로 신경망 SOM을 이용한 일반화 방법인 QSOM 학습법을 제안한다. 이 방법은 기존의 일반화 방법과는 달리 군집화 기능을 제공하는 신경망 SOM을 이용함으로써 명확한 다수의 훈련 예가 없어도 효과적으로 이전에 경험하지 못했던 상태-행동들에 대한 Q값을 예측하고 이용할 수 있다는 장점이 있다. 또한 본 논문에서는 실험을 통해 QSOM 학습법의 일반화 효과와 성능을 평가하였다.

  • PDF

Development of Interior Self-driving Service Robot Using Embedded Board Based on Reinforcement Learning (강화학습 기반 임베디드 보드를 활용한 실내자율 주행 서비스 로봇 개발)

  • Oh, Hyeon-Tack;Baek, Ji-Hoon;Lee, Seung-Jin;Kim, Sang-Hoon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2018.10a
    • /
    • pp.537-540
    • /
    • 2018
  • 본 논문은 Jetson_TX2(임베디드 보드)의 ROS(Robot Operating System)기반으로 맵 지도를 작성하고, SLAM 및 DQN(Deep Q-Network)을 이용한 목적지까지의 이동명령(목표 선속도, 목표 각속도)을 자이로센서로 측정한 현재 각속도를 이용하여 Cortex-M3의 기반의 MCU(Micro Controllor Unit)에 하달하여 엔코더(encoder) 모터에서 측정한 현재 선속도와 자이로센서에서 측정한 각속도 값을 이용하여 PID제어를 통한 실내 자율주행 서비스 로봇.

Developing Mathematics Concepts through Discourses in a Math Classroom (수학수업에서의 담론을 통한 수학적 개념 형성에 관한 연구)

  • Choi-Koh, Sang-Sook;Kang, Hyun-Hee
    • The Mathematical Education
    • /
    • v.46 no.4
    • /
    • pp.423-443
    • /
    • 2007
  • Based on the framework of Huffered-Ackles, Fuson and Sherin(2004), data were analyzed in terms of 3 components: explaining(E), questioning(Q) and justifying(J) of students' mathematical concepts and problem solving in a math classroom. The students used varied presentations to explain and justify their mathematical concepts and ideas. They corrected their mathematical errors or misconceptions through discourses. In addition, they constructed and clarified their concepts and thinking while they were interacted. We were able to recognize there was a special feature in discourses that encouraged the students to construct and develop their mathematical concepts. As they participated in math class and received feedback on their learning, the whole class worked cooperatively in a positive way. Their discourse was improved from the level of the actual development to the level of the potential development and the pattern of interaction moved from ERE(Elicitaion-Response-Elaboration to PD(Proposition Discussion).

  • PDF

RBFN-based Policy Model for Efficient Multiagent Reinforcement Learning (효율적인 멀티 에이전트 강화학습을 위한 RBFN 기반 정책 모델)

  • Gwon, Gi-Deok;Kim, In-Cheol
    • Proceedings of the Korea Inteligent Information System Society Conference
    • /
    • 2007.11a
    • /
    • pp.294-302
    • /
    • 2007
  • 멀티 에이전트 강화학습에서 중요한 이슈 중의 하나는 자신의 성능에 영향을 미칠 수 있는 다른 에이전트들이 존재하는 동적 환경에서 어떻게 최적의 행동 정책을 학습하느냐 하는 것이다. 멀티 에이전트 강화 학습을 위한 기존 연구들은 대부분 단일 에이전트 강화 학습기법들을 큰 변화 없이 그대로 적용하거나 비록 다른 에이전트에 관한 별도의 모델을 이용하더라도 현실적이지 못한 가정들을 요구한다. 본 논문에서는 상대 에이전트에 대한RBFN기반의 행동 정책 모델을 소개한 뒤, 이것을 이용한 강화 학습 방법을 설명한다. 본 논문에서는 제안하는 멀티 에이전트 강화학습 방법은 기존의 멀티 에이전트 강화 학습 연구들과는 달리 상대 에이전트의 Q 평가 함수 모델이 아니라 RBFN 기반의 행동 정책 모델을 학습한다. 또한, 표현력은 풍부하나 학습에 시간과 노력이 많이 요구되는 유한 상태 오토마타나 마코프 체인과 같은 행동 정책 모델들에 비해 비교적 간단한 형태의 행동 정책 모델을 이용함으로써 학습의 효율성을 높였다. 본 논문에서는 대표적이 절대적 멀티 에이전트 환경인 고양이와 쥐 게임을 소개한 뒤, 이 게임을 테스트 베드 삼아 실험들을 전개함으로써 제안하는 RBFN 기반의 정책 모델의 효과를 분석해본다.

  • PDF

Reliability-aware service chaining mapping in NFV-enabled networks

  • Liu, Yicen;Lu, Yu;Qiao, Wenxin;Chen, Xingkai
    • ETRI Journal
    • /
    • v.41 no.2
    • /
    • pp.207-223
    • /
    • 2019
  • Network function virtualization can significantly improve the flexibility and effectiveness of network appliances via a mapping process called service function chaining. However, the failure of any single virtualized network function causes the breakdown of the entire chain, which results in resource wastage, delays, and significant data loss. Redundancy can be used to protect network appliances; however, when failures occur, it may significantly degrade network efficiency. In addition, it is difficult to efficiently map the primary and backups to optimize the management cost and service reliability without violating the capacity, delay, and reliability constraints, which is referred to as the reliability-aware service chaining mapping problem. In this paper, a mixed integer linear programming formulation is provided to address this problem along with a novel online algorithm that adopts the joint protection redundancy model and novel backup selection scheme. The results show that the proposed algorithm can significantly improve the request acceptance ratio and reduce the consumption of physical resources compared to existing backup algorithms.

Hyper-parameter Optimization for Monte Carlo Tree Search using Self-play

  • Lee, Jin-Seon;Oh, Il-Seok
    • Smart Media Journal
    • /
    • v.9 no.4
    • /
    • pp.36-43
    • /
    • 2020
  • The Monte Carlo tree search (MCTS) is a popular method for implementing an intelligent game program. It has several hyper-parameters that require an optimization for showing the best performance. Due to the stochastic nature of the MCTS, the hyper-parameter optimization is difficult to solve. This paper uses the self-playing capability of the MCTS-based game program for optimizing the hyper-parameters. It seeks a winner path over the hyper-parameter space while performing the self-play. The top-q longest winners in the winner path compete for the final winner. The experiment using the 15-15-5 game (Omok in Korean name) showed a promising result.

Process for Automatic Requirement Generation in Korean Requirements Documents using NLP Machine Learning (NLP 기계 학습을 사용한 한글 요구사항 문서에서의 요구사항 자동 생성 프로세스)

  • Young Yun Baek;Soo Jin Park;Young Bum Park
    • Journal of the Semiconductor & Display Technology
    • /
    • v.22 no.1
    • /
    • pp.88-93
    • /
    • 2023
  • In software engineering, requirement analysis is an important task throughout the process and takes up a high proportion. However, factors that fail to analyze requirements include communication failure, different understanding of the meaning of requirements, and failure to perform requirements normally. To solve this problem, we derived actors and behaviors using morpheme analysis and BERT algorithms in the Korean requirement document and constructed them as ontologies. A chatbot system with ontology data is constructed to derive a final system event list through Q&A with users. The chatbot system generates the derived system event list as a requirement diagram and a requirement specification and provides it to the user. Through the above system, diagrams and specifications with a level of coverage complied with Korean requirement documents were created.

  • PDF