• 제어로봇시스템학회:학술대회논문집
• /
• 2001.10a
• /
• pp.164.1-164
• /
• 2001

Q-learning is a recent reinforcement learning algorithm that does not need a modeling of environment and it is a suitable approach to learn behaviors for autonomous agents. But when it is applied to multi-agent learning with many I/O states, it is usually too complex and slow. To overcome this problem in the multi-agent learning system, we propose the successive Q-learning algorithm. Successive Q-learning algorithm divides state-action pairs, which agents can have, into several Q-functions, so it can reduce complexity and calculation amounts. This algorithm is suitable for multi-agent learning in a dynamically changing environment. The proposed successive Q-learning algorithm is applied to the prey-predator problem with the one-prey and two-predators, and its effectiveness is verified from the efficient avoidance ability of the prey agent.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2011.01a
• /
• pp.245-248
• /
• 2011

현재의 영상 검색 시스템은 일반적으로 카테고리 검색 및 분류 검색으로 구성되어 있다. 일반적인 영상 데이터베이스 구축 및 현재의 검색 방법으로는 영상에 대한 재미요인을 분석하여 사용자에게 제공되지 않는다. 하지만 본 논문에서 제시하는 Q-방법론을 사용하여 영상을 분석하였다. Q-방법론에 의하여 분석된 영상은 영상 서버에 저장되며 영상 위치와 분석된 영상 디스크립션 및 재미요인은 데이터베이스에 구축하였다. 또한, 카테고리 검색 및 분류 검색에 대한 단점을 보강하기 위하여 키워드 검색시 색인 사전을 참조하여 온톨로지 검색에 대한 기능을 강화하였다.

• Journal of KSNVE
• /
• v.8 no.5
• /
• pp.964-973
• /
• 1998

In this paper we propose a levitation and imbalance compensation controller design methodology of magnetic bearing system. In order to achieve levitation and elimination of unbalance vibartion in some operation speed we use the discrete-time Q-parameterization control. When rotor speed p = 0 there are no rotor unbalance, with frequency equals to the rotational speed. So in order to make levitatiom we choose the Q-parameterization controller free parameter Q such that the controller has poles on the unit circle at z = 1. However, when rotor speed p $\neq$ 0 there exist sinusoidal disturbance forces, with frequency equals to the rotational speed. So in order to achieve asymptotic rejection of these disturbance forces, the Q-parameterization controller free parameter Q is chosen such that the controller has poles on the unit circle at z = $exp^{ipTs}$ for a certain speed of rotation p ( $T_s$ is the sampling period). First, we introduce the experimental setup employed in this research. Second, we give a mathematical model for the magnetic bearing in difference equation form. Third, we explain the proposed discrete-time Q-parameterization controller design methodology. The controller free parameter Q is assumed to be a proper stable transfer function. Fourth, we show that the controller free parameter which satisfies the design objectives can be obtained by simply solving a set of linear equations rather than solving a complicated optimization problem. Finally, several simulation and experimental results are obtained to evaluate the proposed controller. The results obtained show the effectiveness of the proposed controller in eliminating the unbalance vibrations at the design speed of rotation.

• Proceedings of the Korean Society for Information Management Conference
• /
• 1995.08a
• /
• pp.37-40
• /
• 1995

본 연구는 자연언어시스템에서 색인어와 탐색어의 특정성에 기인하는 재현율 감소를 극복하기 위한 방법론으로써 탐색어의 확장을 통한 검색효율을 평가하였다. 이를 위하여 우리말 데이터베이스를 대상으로 주제전문가가 자연언어로 작성한 원 질의문 (Q1), 원 질의문에 사용된 탐색어와 데이터베이스내의 색인어간의 유사도를 이용하여 탐색어를 확장한 질의문 (Q2(0.2), Q2(0.3)), 주제전문가인 이용자가 Q1의 의미적인 관계를 고려해서 자연언어로 탐색어를 확장한 질의문 (Q3)을 검색효율면에서 비교하였다. 실험결과, 평균재현율은 Q2(0.2), Q2(0.3), Q3, Q1의 검색의 순이었다. 평균정확율은 Q3, Q2(0.3), Q1, Q2(0.2)검색의 순으로 나타났다.

• The Journal of Korean Association of Computer Education
• /
• v.6 no.4
• /
• pp.23-30
• /
• 2003

This paper proposes a passage retrieval system based on Okapi's BM25 for developing an efficient QA system and evaluates performances of the passage retrieval system. The test collection of TREC Q&A track which is composed of about one million documents was indexed and a hundred queries of TREC Q&A track are used as testing queries. The experimental results shows that the proposed passage retrieval system can reach to 100% recall rate by searching in only 1700 sentences while the conventional document retrieval system have to search about 120 thousands sentences which are about 70 times more than the proposed passage retrieval system.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.16 no.3
• /
• pp.470-475
• /
• 2012

In Q-algorithm of EPCglobal Class-1 Gen-2 RFID system, the initial value of $Q_{fp}$, which is the slot-count parameter, is not defined in the standard. And the values of weight C, which is the parameter for incrementing or decrementing the slot-count size, are not determined. Therefore, if the number of tags is small and we let the initial $Q_{fp}$ be large, the number of empty slot will be large. On the other hand, if we let the initial $Q_{fp}$ be small in spite of many tags, almost all the slots will be collided. Also, if the reader selects an inappropriate weight, there are a lot of empty or collided slots. As a result, the performance will be declined because the frame size does not converge to the optimal point quickly during the query round. In this paper, we propose a scheme to allocate the optimal initial $Q_{fp}$ through the tag number estimation and select the weight based on the slot-count size of current query round.

• KIPS Transactions on Computer and Communication Systems
• /
• v.10 no.10
• /
• pp.269-276
• /
• 2021

Recently, smart sensors are used in various environments, and the implementation of ad-hoc sensor networks (ASNs) is a hot research topic. Unfortunately, traditional sensor network routing algorithms focus on specific control issues, and they can't be directly applied to the ASN operation. In this paper, we propose a new routing protocol by using the Q-learning technology, Main challenge of proposed approach is to extend the life of ASNs through efficient energy allocation while obtaining the balanced system performance. The proposed method enhances the Q-learning effect by considering various environmental factors. When a transmission fails, node penalty is accumulated to increase the successful communication probability. Especially, each node stores the Q value of the adjacent node in its own Q table. Every time a data transfer is executed, the Q values are updated and accumulated to learn to select the optimal routing route. Simulation results confirm that the proposed method can choose an energy-efficient routing path, and gets an excellent network performance compared with the existing ASN routing protocols.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.9 no.8
• /
• pp.1649-1655
• /
• 2005

For systems with coexisting cellular and hot-spot cells in broadband orhogonal frequency and code division multiplexing (OFCDM) systems, a suitable cell search algorithm is proposed fur the common pilot channel (CPICH) in the forward link using separated I/Q channel cell specific codes(CSSC), in which the cellular cell specific scrambling code (CCSSC) is assigned to the in-phase (Q) pilot channel of all cellular cells, and the exclusive hot-spot cell specific scrambling code (HSCSSC) group is assigned to the quadrature (Q) pilot channel of all hot-spot cells. Therefore, the proposed algorithm enables a mobile station (MS) to search quickly for the most desirable hot-spot cell due to reducing the effect of CCSSC, when a MS wants to use a mobile internet. The computer simulation results show that the proposed cell search algorithm can achieve faster cell search time performance, compared to conventional cell search methods.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.31 no.3C
• /
• pp.237-244
• /
• 2006

I/Q phase unbalance caused by non-ideal circuit components is inevitable physical phenomenons and leads to performance degradation when we implement a practical coherent M-ary phase shift keying(M-PSK) demodulator. In this paper, we present an exact and general expression involving two-dimensional Gaussian Q-functions for the bit error rate(BER) of uniform M-PSK with I/Q phase unbalance over an additive white Gaussian noise(AWGN) channel. First we derive a BER expression for the k-th bit of 8, 16-PSK signal constellations when Gray code bit mapping is employed. Then, from the derived k-th bit BER expression, we present the exact and general average BER expression for M-PSK with I/Q phase unbalance. This result can readily be applied to numerical evaluation for various cases of practical interest in an I/Q unbalanced M-PSK system, because the one- and two-dimensional Gaussian Q-functions can be easily and directly computed using commonly available mathematical software tools.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.13 no.6
• /
• pp.635-642
• /
• 2003

Reinforcement learning got successful results in a lot of applications such as control and scheduling. Various function approximation methods have been studied in order to improve the learning speed and to solve the shortage of storage in the standard reinforcement learning algorithm of Q-Learning. Most function approximation methods remove some special quality of reinforcement learning and need prior knowledge and preprocessing. Fuzzy Q-Learning needs preprocessing to define fuzzy variables and Local Weighted Regression uses training examples. In this paper, we propose a function approximation method, Fuzzy Q-Map that is based on on-line fuzzy clustering. Fuzzy Q-Map classifies a query state and predicts a suitable action according to the membership degree. We applied the Fuzzy Q-Map, CMAC and LWR to the mountain car problem. Fuzzy Q-Map reached the optimal prediction rate faster than CMAC and the lower prediction rate was seen than LWR that uses training example.