• The KIPS Transactions:PartB
• /
• v.10B no.3
• /
• pp.237-242
• /
• 2003

Ant Colony System (ACS) Algorithm is new meta heuristic for hard combinatorial optimization problem. It is a population based approach that uses exploitation of positive feedback as well as greedy search. It was first proposed for tackling the well known Traveling Salesman Problem (TSP). In this paper, we introduce ACS of new method that adds reinforcement value for each edge that visit to Local/Global updating rule. and the performance results under various conditions are conducted, and the comparision between the original ACS and the proposed method is shown. It turns out that our proposed method can compete with tile original ACS in terms of solution quality and computation speed to these problem.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.16 no.1
• /
• pp.334-349
• /
• 2022

The effectiveness of Wi-Fi networks is greatly influenced by the optimization of contention window (CW) parameters. Unfortunately, the conventional approach employed by IEEE 802.11 wireless networks is not scalable enough to sustain consistent performance for the increasing number of stations. Yet, it is still the default when accessing channels for single-users of 802.11 transmissions. Recently, there has been a spike in attempts to enhance network performance using a machine learning (ML) technique known as reinforcement learning (RL). Its advantage is interacting with the surrounding environment and making decisions based on its own experience. Deep RL (DRL) uses deep neural networks (DNN) to deal with more complex environments (such as continuous state spaces or actions spaces) and to get optimum rewards. As a result, we present a new approach of CW control mechanism, which is termed as contention window threshold (CW_Threshold). It uses the DRL principle to define the threshold value and learn optimal settings under various network scenarios. We demonstrate our proposed method, known as a smart exponential-threshold-linear backoff algorithm with a deep Q-learning network (SETL-DQN). The simulation results show that our proposed SETL-DQN algorithm can effectively improve the throughput and reduce the collision rates.

• International Journal of Internet, Broadcasting and Communication
• /
• v.16 no.3
• /
• pp.205-211
• /
• 2024

Effective warehouse management requires advanced resource planning to optimize profits and space. Robots offer a promising solution, but their effectiveness relies on embedded artificial intelligence. Multi-agent reinforcement learning (MARL) enhances robot intelligence in these environments. This study explores various MARL algorithms using the Multi-Robot Warehouse Environment (RWARE) to determine their suitability for warehouse resource planning. Our findings show that cooperative MARL is essential for effective warehouse management. IA2C outperforms MAA2C and VDA2C on smaller maps, while VDA2C excels on larger maps. IA2C's decentralized approach, focusing on cooperation over collaboration, allows for higher reward collection in smaller environments. However, as map size increases, reward collection decreases due to the need for extensive exploration. This study highlights the importance of selecting the appropriate MARL algorithm based on the specific warehouse environment's requirements and scale.

• ETRI Journal
• /
• v.44 no.5
• /
• pp.733-745
• /
• 2022

In software-defined wireless networking (SDWN), the optimal routing technique is one of the effective solutions to improve its performance. This routing technique is done by many different methods, with the most common using integer linear programming problem (ILP), building optimal routing metrics. These methods often only focus on one routing objective, such as minimizing the packet blocking probability, minimizing end-to-end delay (EED), and maximizing network throughput. It is difficult to consider multiple objectives concurrently in a routing algorithm. In this paper, we investigate the application of machine learning to control routing in the SDWN. An intelligent routing algorithm is then proposed based on the machine learning to improve the network performance. The proposed algorithm can optimize multiple routing objectives. Our idea is to combine supervised learning (SL) and reinforcement learning (RL) methods to discover new routes. The SL is used to predict the performance metrics of the links, including EED quality of transmission (QoT), and packet blocking probability (PBP). The routing is done by the RL method. We use the Q-value in the fundamental equation of the RL to store the PBP, which is used for the aim of route selection. Concurrently, the learning rate coefficient is flexibly changed to determine the constraints of routing during learning. These constraints include QoT and EED. Our performance evaluations based on OMNeT++ have shown that the proposed algorithm has significantly improved the network performance in terms of the QoT, EED, packet delivery ratio, and network throughput compared with other well-known routing algorithms.

• Journal of the Earthquake Engineering Society of Korea
• /
• v.28 no.2
• /
• pp.113-119
• /
• 2024

Existing reinforced concrete buildings with seismically deficient column details affect the overall behavior depending on the failure type of column. This study aims to develop and validate a machine learning-based prediction model for the column failure modes (shear, flexure-shear, and flexure failure modes). For this purpose, artificial neural network (ANN), K-nearest neighbor (KNN), decision tree (DT), and random forest (RF) models were used, considering previously collected experimental data. Using four machine learning methodologies, we developed a classification learning model that can predict the column failure modes in terms of the input variables using concrete compressive strength, steel yield strength, axial load ratio, height-to-dept aspect ratio, longitudinal reinforcement ratio, and transverse reinforcement ratio. The performance of each machine learning model was compared and verified by calculating accuracy, precision, recall, F1-Score, and ROC. Based on the performance measurements of the classification model, the RF model represents the highest average value of the classification model performance measurements among the considered learning methods, and it can conservatively predict the shear failure mode. Thus, the RF model can rapidly predict the column failure modes with simple column details.

• The KIPS Transactions:PartB
• /
• v.10B no.6
• /
• pp.587-592
• /
• 2003

Many real world control problems have continuous states and actions. When the state space is continuous, the reinforcement learning problems involve very large state space and suffer from memory and time for learning all individual state-action values. These problems need function approximators that reason action about new state from previously experienced states. We introduce Fuzzy Q-Map that is a function approximators for 1 - step Q-learning and is based on fuzzy clustering. Fuzzy Q-Map groups similar states and chooses an action and refers Q value according to membership degree. The centroid and Q value of winner cluster is updated using membership degree and TD(Temporal Difference) error. We applied Fuzzy Q-Map to the mountain car problem and acquired accelerated learning speed.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2021.05a
• /
• pp.447-450
• /
• 2021

시뮬레이션을 비롯한 많은 다중 에이전트 환경에서는 중앙 집중 훈련 및 분산 수행(centralized training with decentralized execution; CTDE) 방식이 활용되고 있다. CTDE 방식 하에서 중앙 집중 훈련 및 분산 수행 환경에서의 다중 에이전트 학습을 위한 상태 행동 가치 기반(state-action value; Q-value) 다중 에이전트 알고리즘들에 대한 많은 연구가 이루어졌다. 이러한 알고리즘들은 Independent Q-learning (IQL)이라는 강력한 벤치 마크 알고리즘에서 파생되어 다중 에이전트의 공동의 상태 행동 가치의 분해(Decomposition) 문제에 대해 집중적으로 연구되었다. 본 논문에서는 앞선 연구들에 관한 알고리즘들에 대한 분석과 실용적이고 일반적인 도메인에서의 실험 분석을 통해 검증한다.

• International Journal of Advanced Culture Technology
• /
• v.8 no.1
• /
• pp.128-134
• /
• 2020

The purpose of this study is to investigate the role and function of tokens to form a healthy blockchain-based ecosystem. Tokens must be constructed in a way that enhances their desired behavior to grow into a healthy token economy. The actions required of ecosystem participants in designing tokens should enable each individual to receive appropriate incentives (rewards) and encourage voluntary participation in taking this action. Also, all ecosystem participants must design to make the token ecosystem self-sustainable by generating profits. For example, in Bitcoin's proof-of-work method, mining is designed as a desirable behavior. Token-based services should be designed to induce multiple engagements, to design penalties for undesirable behavior, and to take into account evolutionary development potentials. Besides, the economic value of the entire token ecosystem will increase if the value that is designed and designed to take into account the revolutionary Innovation Possibility is greater than the reward amount paid to tokens. This study will contribute to presenting relevant service model by presenting how to design tokens and criteria when establishing blockchain-based service model. Future research is needed to discover new facts through a detailed comparative analysis between Tokennomics models.

• Steel and Composite Structures
• /
• v.8 no.6
• /
• pp.457-473
• /
• 2008

Yielding of the internal steel reinforcement is an important mechanism that influences the Intermediate Crack-induced debonding (IC debonding) behavior in FRP-strengthened RC members since the FRP is required to carry additional forces beyond the condition of steel yielding. However, rational design practice dictates an appropriate limit state is defined when steel yielding is assured prior to FRP debonding. This paper proposes a criterion which correlates the occurrence of IC debonding to the formulation of a critical steel yielding length. Once this length is exceeded the average bond stress in the FRP/concrete interface exceeds its threshold value, which proves to correlate with the average bond resistance in an FRP/concrete joint under simple shear loading. This proposed IC debonding concept is based on traditional sections analysis which is conventionally applied in design practice. Hence complex bond stress-slip analyses are avoided. Furthermore, the proposed model incorporates not only the bond properties of FRP/concrete interface but also the beam geometry, and properties of steel and FRP reinforcement in the analysis of IC debonding strength. Based upon a solid database, the validity of the proposed simple IC debonding criterion is demonstrated.

• Proceedings of the Korea Inteligent Information System Society Conference
• /
• 2007.11a
• /
• pp.284-293
• /
• 2007

본 논문에서는 실시간 동정 환경에 효과적인 L-CAA 에이전트 구조를 제안한다. L-CAA 에이전트 구조는 변화하는 환경에 대한 적응성을 높이기 위해, 선행 연구를 통해 개발된 행위 기반 에이전트 구조인 CAA에 강화 학습 기능을 추가하여 확장한 것이다. 안정적인 성능을 위해 L-CAA에서 행위 선택 메커니즘은 크게 두 단계로 나뉜다. 첫 번째 단계에서는 사용자가 미리 정의한 각 행위의 수행 가능 조건과 효용성을 검사함으로써 행위 라이브러리로부터 실행할 행위들을 추출한다. 하지만 첫 번째 단계에서 다수의 행위가 추출되면, 두 번째 단계에서는 강화 학습의 도움을 받아 이들 중에서 실행할 하나의 행위를 선택한다. 즉, 강화 학습을 통해 갱신된 각 행위들의 Q 함수 값을 서로 비교함으로써, 가장 큰 기대 보상 값을 가진 행위를 선택하여 실행한다. 또한 L-CAA에서는 실행 중인 행위의 유지 가능 조건을 지속적으로 검사하여 환경의 동적 변화로 인해 일부 조건이 만족되지 않는 경우가 발생하면 현재 행위의 실행을 즉시 종료할 수 있다. 그 뿐 아니라, L-CAA는 행위 실행 중에도 효용성이 더 높은 다른 행위가 발생하면 현재의 행위를 일시 정지하였다가 복귀하는 기능도 제공한다. 본 논문에서는 L-CAA 구조의 효과를 분석하기 위해, 대표적인 동적 가상환경인 Unreal Tournament 게임에서 자율적을 동작하는 L-CAA기반의 UTBot 들을 구현하고, 이들을 이용하여 성능실험을 전개해본다.