• Title/Summary/Keyword: sequence-to-sequence learning

Search Result 424, Processing Time 0.04 seconds

Intrusion Detection Method Using Unsupervised Learning-Based Embedding and Autoencoder (비지도 학습 기반의 임베딩과 오토인코더를 사용한 침입 탐지 방법)

  • Junwoo Lee;Kangseok Kim
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.12 no.8
    • /
    • pp.355-364
    • /
    • 2023
  • As advanced cyber threats continue to increase in recent years, it is difficult to detect new types of cyber attacks with existing pattern or signature-based intrusion detection method. Therefore, research on anomaly detection methods using data learning-based artificial intelligence technology is increasing. In addition, supervised learning-based anomaly detection methods are difficult to use in real environments because they require sufficient labeled data for learning. Research on an unsupervised learning-based method that learns from normal data and detects an anomaly by finding a pattern in the data itself has been actively conducted. Therefore, this study aims to extract a latent vector that preserves useful sequence information from sequence log data and develop an anomaly detection learning model using the extracted latent vector. Word2Vec was used to create a dense vector representation corresponding to the characteristics of each sequence, and an unsupervised autoencoder was developed to extract latent vectors from sequence data expressed as dense vectors. The developed autoencoder model is a recurrent neural network GRU (Gated Recurrent Unit) based denoising autoencoder suitable for sequence data, a one-dimensional convolutional neural network-based autoencoder to solve the limited short-term memory problem that GRU can have, and an autoencoder combining GRU and one-dimensional convolution was used. The data used in the experiment is time-series-based NGIDS (Next Generation IDS Dataset) data, and as a result of the experiment, an autoencoder that combines GRU and one-dimensional convolution is better than a model using a GRU-based autoencoder or a one-dimensional convolution-based autoencoder. It was efficient in terms of learning time for extracting useful latent patterns from training data, and showed stable performance with smaller fluctuations in anomaly detection performance.

Development of Combined Architecture of Multiple Deep Convolutional Neural Networks for Improving Video Face Identification (비디오 얼굴 식별 성능개선을 위한 다중 심층합성곱신경망 결합 구조 개발)

  • Kim, Kyeong Tae;Choi, Jae Young
    • Journal of Korea Multimedia Society
    • /
    • v.22 no.6
    • /
    • pp.655-664
    • /
    • 2019
  • In this paper, we propose a novel way of combining multiple deep convolutional neural network (DCNN) architectures which work well for accurate video face identification by adopting a serial combination of 3D and 2D DCNNs. The proposed method first divides an input video sequence (to be recognized) into a number of sub-video sequences. The resulting sub-video sequences are used as input to the 3D DCNN so as to obtain the class-confidence scores for a given input video sequence by considering both temporal and spatial face feature characteristics of input video sequence. The class-confidence scores obtained from corresponding sub-video sequences is combined by forming our proposed class-confidence matrix. The resulting class-confidence matrix is then used as an input for learning 2D DCNN learning which is serially linked to 3D DCNN. Finally, fine-tuned, serially combined DCNN framework is applied for recognizing the identity present in a given test video sequence. To verify the effectiveness of our proposed method, extensive and comparative experiments have been conducted to evaluate our method on COX face databases with their standard face identification protocols. Experimental results showed that our method can achieve better or comparable identification rate compared to other state-of-the-art video FR methods.

The Effect of Implicit Motor Sequence Learning Through Perceptual-Motor Task in Patients with Subacute Stroke (아급성기 뇌졸중 환자에서 지각-운동 과제를 통한 내잠 학습의 효과)

  • Lee, Mi-Young;Park, Rae-Joon;Nam, Ki-Seok
    • The Journal of Korean Physical Therapy
    • /
    • v.20 no.3
    • /
    • pp.1-7
    • /
    • 2008
  • Purpose: Implicit motor learning is the capacity to acquire skill through physical practice without conscious awareness of what elements of performance improved. This study investigated whether subacute stroke patients can implicitly learn a perceptual-motor task. Methods: We recruited 12 patients with subacute stroke and 12 age-matched controls. All participants performed a perceptual-motor task that involved pressing a button corresponding with colored circles (blue, green, yellow, red) on a computer screen. The task consists of 7 blocks composed of 10 repetitions for a repeating 12-element sequence (total 120 responses). Results: Both groups demonstrated significant improvement in acquisition performance. Reaction times deceased in both groups at similar rate within the sequential block trials (2-5 blocks), and reaction times increased at a similar rate when the task paradigm was transferred from the sequential block trial to the random block trial (5-6-7 blocks). Conclusion: The results of this study suggest that patients with sub-actue stroke can implicitly learn a perceptual motor skill. Although explicit instructions should be used to focus the learner's attention rather than provide information about the task, the application of implicit motor learning strategies in the rehabilitation setting may be beneficial.

  • PDF

Uncertainty Sequence Modeling Approach for Safe and Effective Autonomous Driving (안전하고 효과적인 자율주행을 위한 불확실성 순차 모델링)

  • Yoon, Jae Ung;Lee, Ju Hong
    • Smart Media Journal
    • /
    • v.11 no.9
    • /
    • pp.9-20
    • /
    • 2022
  • Deep reinforcement learning(RL) is an end-to-end data-driven control method that is widely used in the autonomous driving domain. However, conventional RL approaches have difficulties in applying it to autonomous driving tasks due to problems such as inefficiency, instability, and uncertainty. These issues play an important role in the autonomous driving domain. Although recent studies have attempted to solve these problems, they are computationally expensive and rely on special assumptions. In this paper, we propose a new algorithm MCDT that considers inefficiency, instability, and uncertainty by introducing a method called uncertainty sequence modeling to autonomous driving domain. The sequence modeling method, which views reinforcement learning as a decision making generation problem to obtain high rewards, avoids the disadvantages of exiting studies and guarantees efficiency, stability and also considers safety by integrating uncertainty estimation techniques. The proposed method was tested in the OpenAI Gym CarRacing environment, and the experimental results show that the MCDT algorithm provides efficient, stable and safe performance compared to the existing reinforcement learning method.

A Python-based educational software tool for visualizing bioinformatics alignment algorithms

  • Elis Khatizah;Hee-Jo Nam;Hyun-Seok Park
    • Genomics & Informatics
    • /
    • v.21 no.1
    • /
    • pp.15.1-15.4
    • /
    • 2023
  • Bioinformatics education can be defined as the teaching and learning of how to use software tools, along with mathematical and statistical analysis, to solve biological problems. Although many resources are available, most students still struggle to understand even the simplest sequence alignment algorithms. Applying visualizations to these topics benefits both lecturers and students. Unfortunately, educational software for visualizing step-by-step processes in the user experience of sequence alignment algorithms is rare. In this article, an educational visualization tool for biological sequence alignment is presented, and the source code is released in order to encourage the collaborative power of open-source software, with the expectation of further contributions from the community in the future. Two different modules are integrated to enable a student to investigate the characteristics of alignment algorithms.

An Efficient and Accurate Artificial Neural Network through Induced Learning Retardation and Pruning Training Methods Sequence

  • Bandibas, Joel;Kohyama, Kazunori;Wakita, Koji
    • Proceedings of the KSRS Conference
    • /
    • 2003.11a
    • /
    • pp.429-431
    • /
    • 2003
  • The induced learning retardation method involves the temporary inhibition of the artificial neural network’s active units from participating in the error reduction process during training. This stimulates the less active units to contribute significantly to reduce the network error. However, some less active units are not sensitive to stimulation making them almost useless. The network can then be pruned by removing the less active units to make it smaller and more efficient. This study focuses on making the network more efficient and accurate by developing the induced learning retardation and pruning sequence training method. The developed procedure results to faster learning and more accurate artificial neural network for satellite image classification.

  • PDF

Two-step Document Summarization using Deep Learning and Maximal Marginal Relevance (딥러닝과 Maximal Marginal Relevance를 이용한 2단계 문서 요약)

  • Jeon, Jaewon;Hwang, Hyunsun;Lee, Changki
    • Annual Conference on Human and Language Technology
    • /
    • 2019.10a
    • /
    • pp.297-300
    • /
    • 2019
  • 문서 요약은 길이가 긴 원본 문서의 의미는 유지한 채 원본보다 짧은 문서나 문장을 생성하는 자연어 처리 태스크이다. 본 논문에서는 Maximal Marginal Relevance(MMR)를 이용한 sequence-to-sequence 문장 추출 모델을 이용하여 의미가 중복되는 문장을 최소화하는 문장을 추출하고 추출된 문장을 sequence-to-sequence 모델을 통해 요약문을 생성하는 2단계 문서 요약 모델을 제안한다. 실험 결과 MMR을 활용하지 않았던 기존의 방법론보다 Rouge 성능이 향상되었다.

  • PDF

Prospective Changes of English Digital Textbook Based on the Universal Design for Learning (보편적 학습 설계에 근거한 영어과 디지털 교과서 개선 방안)

  • Kim, Jeong-ryeol
    • The Journal of the Korea Contents Association
    • /
    • v.15 no.7
    • /
    • pp.674-683
    • /
    • 2015
  • One of the issues with the textbooks pertinent to the current study is whether or not the Universal Design for Learning (UDL) factors have been dealt to satisfy students with different aptitudes in learning the core objectives of the lessons. This study develops a modified version of the UDL analysis criteria from the cross curricular criteria to language teaching and learning and uses it to analyze the sequence of digital English textbooks to investigate the descriptive statistics of the UDL factors in the new textbooks. The result shows that the textbook is designed most favorably to the students with the talent of linguistic aptitude and less favorably to the students with other types of aptitudes. The sequence analysis shows that sentence/word length and appearance of new words are incrementally sequenced as students advance upper grades. However, the syntactic complexity of middle school curves up steeply which is different from the elementary school textbooks. The UDL analysis will provide learning factors to consider when designing digital English textbooks to cover different aptitudinal groups.

Improvements in Patch-Based Machine Learning for Analyzing Three-Dimensional Seismic Sequence Data (3차원 탄성파자료의 층서구분을 위한 패치기반 기계학습 방법의 개선)

  • Lee, Donguk;Moon, Hye-Jin;Kim, Chung-Ho;Moon, Seonghoon;Lee, Su Hwan;Jou, Hyeong-Tae
    • Geophysics and Geophysical Exploration
    • /
    • v.25 no.2
    • /
    • pp.59-70
    • /
    • 2022
  • Recent studies demonstrate that machine learning has expanded in the field of seismic interpretation. Many convolutional neural networks have been developed for seismic sequence identification, which is important for seismic interpretation. However, expense and time limitations indicate that there is insufficient data available to provide a sufficient dataset to train supervised machine learning programs to identify seismic sequences. In this study, patch division and data augmentation are applied to mitigate this lack of data. Furthermore, to obtain spatial information that could be lost during patch division, an artificial channel is added to the original data to indicate depth. Seismic sequence identification is performed using a U-Net network and the Netherlands F3 block dataset from the dGB Open Seismic Repository, which offers datasets for machine learning, and the predicted results are evaluated. The results show that patch-based U-Net seismic sequence identification is improved by data augmentation and the addition of an artificial channel.

Survey on Nucleotide Encoding Techniques and SVM Kernel Design for Human Splice Site Prediction

  • Bari, A.T.M. Golam;Reaz, Mst. Rokeya;Choi, Ho-Jin;Jeong, Byeong-Soo
    • Interdisciplinary Bio Central
    • /
    • v.4 no.4
    • /
    • pp.14.1-14.6
    • /
    • 2012
  • Splice site prediction in DNA sequence is a basic search problem for finding exon/intron and intron/exon boundaries. Removing introns and then joining the exons together forms the mRNA sequence. These sequences are the input of the translation process. It is a necessary step in the central dogma of molecular biology. The main task of splice site prediction is to find out the exact GT and AG ended sequences. Then it identifies the true and false GT and AG ended sequences among those candidate sequences. In this paper, we survey research works on splice site prediction based on support vector machine (SVM). The basic difference between these research works is nucleotide encoding technique and SVM kernel selection. Some methods encode the DNA sequence in a sparse way whereas others encode in a probabilistic manner. The encoded sequences serve as input of SVM. The task of SVM is to classify them using its learning model. The accuracy of classification largely depends on the proper kernel selection for sequence data as well as a selection of kernel parameter. We observe each encoding technique and classify them according to their similarity. Then we discuss about kernel and their parameter selection. Our survey paper provides a basic understanding of encoding approaches and proper kernel selection of SVM for splice site prediction.