• Title/Summary/Keyword: Training intelligence

Search Result 750, Processing Time 0.025 seconds

An Intelligent Intrusion Detection Model Based on Support Vector Machines and the Classification Threshold Optimization for Considering the Asymmetric Error Cost (비대칭 오류비용을 고려한 분류기준값 최적화와 SVM에 기반한 지능형 침입탐지모형)

  • Lee, Hyeon-Uk;Ahn, Hyun-Chul
    • Journal of Intelligence and Information Systems
    • /
    • v.17 no.4
    • /
    • pp.157-173
    • /
    • 2011
  • As the Internet use explodes recently, the malicious attacks and hacking for a system connected to network occur frequently. This means the fatal damage can be caused by these intrusions in the government agency, public office, and company operating various systems. For such reasons, there are growing interests and demand about the intrusion detection systems (IDS)-the security systems for detecting, identifying and responding to unauthorized or abnormal activities appropriately. The intrusion detection models that have been applied in conventional IDS are generally designed by modeling the experts' implicit knowledge on the network intrusions or the hackers' abnormal behaviors. These kinds of intrusion detection models perform well under the normal situations. However, they show poor performance when they meet a new or unknown pattern of the network attacks. For this reason, several recent studies try to adopt various artificial intelligence techniques, which can proactively respond to the unknown threats. Especially, artificial neural networks (ANNs) have popularly been applied in the prior studies because of its superior prediction accuracy. However, ANNs have some intrinsic limitations such as the risk of overfitting, the requirement of the large sample size, and the lack of understanding the prediction process (i.e. black box theory). As a result, the most recent studies on IDS have started to adopt support vector machine (SVM), the classification technique that is more stable and powerful compared to ANNs. SVM is known as a relatively high predictive power and generalization capability. Under this background, this study proposes a novel intelligent intrusion detection model that uses SVM as the classification model in order to improve the predictive ability of IDS. Also, our model is designed to consider the asymmetric error cost by optimizing the classification threshold. Generally, there are two common forms of errors in intrusion detection. The first error type is the False-Positive Error (FPE). In the case of FPE, the wrong judgment on it may result in the unnecessary fixation. The second error type is the False-Negative Error (FNE) that mainly misjudges the malware of the program as normal. Compared to FPE, FNE is more fatal. Thus, when considering total cost of misclassification in IDS, it is more reasonable to assign heavier weights on FNE rather than FPE. Therefore, we designed our proposed intrusion detection model to optimize the classification threshold in order to minimize the total misclassification cost. In this case, conventional SVM cannot be applied because it is designed to generate discrete output (i.e. a class). To resolve this problem, we used the revised SVM technique proposed by Platt(2000), which is able to generate the probability estimate. To validate the practical applicability of our model, we applied it to the real-world dataset for network intrusion detection. The experimental dataset was collected from the IDS sensor of an official institution in Korea from January to June 2010. We collected 15,000 log data in total, and selected 1,000 samples from them by using random sampling method. In addition, the SVM model was compared with the logistic regression (LOGIT), decision trees (DT), and ANN to confirm the superiority of the proposed model. LOGIT and DT was experimented using PASW Statistics v18.0, and ANN was experimented using Neuroshell 4.0. For SVM, LIBSVM v2.90-a freeware for training SVM classifier-was used. Empirical results showed that our proposed model based on SVM outperformed all the other comparative models in detecting network intrusions from the accuracy perspective. They also showed that our model reduced the total misclassification cost compared to the ANN-based intrusion detection model. As a result, it is expected that the intrusion detection model proposed in this paper would not only enhance the performance of IDS, but also lead to better management of FNE.

Increasing Accuracy of Stock Price Pattern Prediction through Data Augmentation for Deep Learning (데이터 증강을 통한 딥러닝 기반 주가 패턴 예측 정확도 향상 방안)

  • Kim, Youngjun;Kim, Yeojeong;Lee, Insun;Lee, Hong Joo
    • The Journal of Bigdata
    • /
    • v.4 no.2
    • /
    • pp.1-12
    • /
    • 2019
  • As Artificial Intelligence (AI) technology develops, it is applied to various fields such as image, voice, and text. AI has shown fine results in certain areas. Researchers have tried to predict the stock market by utilizing artificial intelligence as well. Predicting the stock market is known as one of the difficult problems since the stock market is affected by various factors such as economy and politics. In the field of AI, there are attempts to predict the ups and downs of stock price by studying stock price patterns using various machine learning techniques. This study suggest a way of predicting stock price patterns based on the Convolutional Neural Network(CNN) among machine learning techniques. CNN uses neural networks to classify images by extracting features from images through convolutional layers. Therefore, this study tries to classify candlestick images made by stock data in order to predict patterns. This study has two objectives. The first one referred as Case 1 is to predict the patterns with the images made by the same-day stock price data. The second one referred as Case 2 is to predict the next day stock price patterns with the images produced by the daily stock price data. In Case 1, data augmentation methods - random modification and Gaussian noise - are applied to generate more training data, and the generated images are put into the model to fit. Given that deep learning requires a large amount of data, this study suggests a method of data augmentation for candlestick images. Also, this study compares the accuracies of the images with Gaussian noise and different classification problems. All data in this study is collected through OpenAPI provided by DaiShin Securities. Case 1 has five different labels depending on patterns. The patterns are up with up closing, up with down closing, down with up closing, down with down closing, and staying. The images in Case 1 are created by removing the last candle(-1candle), the last two candles(-2candles), and the last three candles(-3candles) from 60 minutes, 30 minutes, 10 minutes, and 5 minutes candle charts. 60 minutes candle chart means one candle in the image has 60 minutes of information containing an open price, high price, low price, close price. Case 2 has two labels that are up and down. This study for Case 2 has generated for 60 minutes, 30 minutes, 10 minutes, and 5minutes candle charts without removing any candle. Considering the stock data, moving the candles in the images is suggested, instead of existing data augmentation techniques. How much the candles are moved is defined as the modified value. The average difference of closing prices between candles was 0.0029. Therefore, in this study, 0.003, 0.002, 0.001, 0.00025 are used for the modified value. The number of images was doubled after data augmentation. When it comes to Gaussian Noise, the mean value was 0, and the value of variance was 0.01. For both Case 1 and Case 2, the model is based on VGG-Net16 that has 16 layers. As a result, 10 minutes -1candle showed the best accuracy among 60 minutes, 30 minutes, 10 minutes, 5minutes candle charts. Thus, 10 minutes images were utilized for the rest of the experiment in Case 1. The three candles removed from the images were selected for data augmentation and application of Gaussian noise. 10 minutes -3candle resulted in 79.72% accuracy. The accuracy of the images with 0.00025 modified value and 100% changed candles was 79.92%. Applying Gaussian noise helped the accuracy to be 80.98%. According to the outcomes of Case 2, 60minutes candle charts could predict patterns of tomorrow by 82.60%. To sum up, this study is expected to contribute to further studies on the prediction of stock price patterns using images. This research provides a possible method for data augmentation of stock data.

  • PDF

Diagnostic Classification of Chest X-ray Pneumonia using Inception V3 Modeling (Inception V3를 이용한 흉부촬영 X선 영상의 폐렴 진단 분류)

  • Kim, Ji-Yul;Ye, Soo-Young
    • Journal of the Korean Society of Radiology
    • /
    • v.14 no.6
    • /
    • pp.773-780
    • /
    • 2020
  • With the development of the 4th industrial, research is being conducted to prevent diseases and reduce damage in various fields of science and technology such as medicine, health, and bio. As a result, artificial intelligence technology has been introduced and researched for image analysis of radiological examinations. In this paper, we will directly apply a deep learning model for classification and detection of pneumonia using chest X-ray images, and evaluate whether the deep learning model of the Inception series is a useful model for detecting pneumonia. As the experimental material, a chest X-ray image data set provided and shared free of charge by Kaggle was used, and out of the total 3,470 chest X-ray image data, it was classified into 1,870 training data sets, 1,100 validation data sets, and 500 test data sets. I did. As a result of the experiment, the result of metric evaluation of the Inception V3 deep learning model was 94.80% for accuracy, 97.24% for precision, 94.00% for recall, and 95.59 for F1 score. In addition, the accuracy of the final epoch for Inception V3 deep learning modeling was 94.91% for learning modeling and 89.68% for verification modeling for pneumonia detection and classification of chest X-ray images. For the evaluation of the loss function value, the learning modeling was 1.127% and the validation modeling was 4.603%. As a result, it was evaluated that the Inception V3 deep learning model is a very excellent deep learning model in extracting and classifying features of chest image data, and its learning state is also very good. As a result of matrix accuracy evaluation for test modeling, the accuracy of 96% for normal chest X-ray image data and 97% for pneumonia chest X-ray image data was proven. The deep learning model of the Inception series is considered to be a useful deep learning model for classification of chest diseases, and it is expected that it can also play an auxiliary role of human resources, so it is considered that it will be a solution to the problem of insufficient medical personnel. In the future, this study is expected to be presented as basic data for similar studies in the case of similar studies on the diagnosis of pneumonia using deep learning.

A Study on Improvement of the police disaster crisis management system (경찰의 재난위기관리 개선에 관한 연구)

  • Chun, Yongtae;Kim, Moonkwi
    • Journal of the Society of Disaster Information
    • /
    • v.11 no.4
    • /
    • pp.556-569
    • /
    • 2015
  • With about 75% of the population of Korea criticizing the government's disaster policy and a failure to respond to large-scale emergency like the Sewol ferry sinking means that there is a deep distrust in the government. In order to prevent dreadful disasters such as the Sewol ferry sinking, it is important to secure a prime time with respect to disaster safety. Improving crisis management skills and managerial role of police officers who are in close proximity to the people is necessary for the success of disaster management. With disaster management as one of the most essential missions of the police, as a part of a national crisis management, a step by step strengthening of the disaster safety management system of the police is necessary, as below. First, at the prevention phase, law enforcement officers were not injected into for profit large-scale assemblies or events, but in the future the involvement, injection should be based on the level of potential risk, rather than profitability. In the past and now, the priortiy was the priority was on traffic flow, traffic communication, however, the paradigm of traffic policy should be changed to a safety-centered policy. To prevent large-scale accidents, police investigators should root out improper routines and illegal construction subcontracting. The police (intelligence) should strengthen efforts to collect intelligence under the subject of "safety". Second, with respect to the preparatory phase, on a survey of police officers, the result showed that 72% of police officers responded that safety management was not related to the job descriptions of the police. This, along with other results, shows that the awareness of disaster safety must be adopted by, or rather changed in the police urgently. The training in disaster safety education should be strengthened. A network of experts (private, administrative, and police) in safety management should be established to take advantage of private resources with regard to crisis situtions. Third, with respect to the response phase, for rapid first responses to occur, a unified communication network should be established, and a real-time video information network should be adopted by the police and installed in the police situation room. Fourth, during the recovery phase, recovery teams should be injected, added and operated to minimize secondary damage.

Analysis of the Impact of Satellite Remote Sensing Information on the Prediction Performance of Ungauged Basin Stream Flow Using Data-driven Models (인공위성 원격 탐사 정보가 자료 기반 모형의 미계측 유역 하천유출 예측성능에 미치는 영향 분석)

  • Seo, Jiyu;Jung, Haeun;Won, Jeongeun;Choi, Sijung;Kim, Sangdan
    • Journal of Wetlands Research
    • /
    • v.26 no.2
    • /
    • pp.147-159
    • /
    • 2024
  • Lack of streamflow observations makes model calibration difficult and limits model performance improvement. Satellite-based remote sensing products offer a new alternative as they can be actively utilized to obtain hydrological data. Recently, several studies have shown that artificial intelligence-based solutions are more appropriate than traditional conceptual and physical models. In this study, a data-driven approach combining various recurrent neural networks and decision tree-based algorithms is proposed, and the utilization of satellite remote sensing information for AI training is investigated. The satellite imagery used in this study is from MODIS and SMAP. The proposed approach is validated using publicly available data from 25 watersheds. Inspired by the traditional regionalization approach, a strategy is adopted to learn one data-driven model by integrating data from all basins, and the potential of the proposed approach is evaluated by using a leave-one-out cross-validation regionalization setting to predict streamflow from different basins with one model. The GRU + Light GBM model was found to be a suitable model combination for target basins and showed good streamflow prediction performance in ungauged basins (The average model efficiency coefficient for predicting daily streamflow in 25 ungauged basins is 0.7187) except for the period when streamflow is very small. The influence of satellite remote sensing information was found to be up to 10%, with the additional application of satellite information having a greater impact on streamflow prediction during low or dry seasons than during wet or normal seasons.

Study on the Direction of Universal Big Data and Big Data Education-Based on the Survey of Big Data Experts (보편적 빅데이터와 빅데이터 교육의 방향성 연구 - 빅데이터 전문가의 인식 조사를 기반으로)

  • Park, Youn-Soo;Lee, Su-Jin
    • Journal of The Korean Association of Information Education
    • /
    • v.24 no.2
    • /
    • pp.201-214
    • /
    • 2020
  • Big data is gradually expanding in diverse fields, with changing the data-related legislation. Moreover it would be interest in big data education. However, it requires a high level of knowledge and skills in order to utilize Big Data and it takes a long time for education spends a lot of money for training. We study that in order to define Universal Big Data used to the industrial field in a wide range. As a result, we make the paradigm for Big Data education for college students. We survey to the professional the Big Data definition and the Big Data perception. According to the survey, the Big Data related-professional recognize that is a wider definition than Computer Science Big Data is. Also they recognize that the Big Data Processing dose not be required Big Data Processing Frameworks or High Performance Computers. This means that in order to educate Big Data, it is necessary to focus on the analysis methods and application methods of Universal Big Data rather than computer science (Engineering) knowledge and skills. Based on the our research, we propose the Universal Big Data education on the new paradigm.

An analysis of public perception on Artificial Intelligence(AI) education using Big Data: Based on News articles and Twitter (빅데이터 분석을 통해 본 AI교육에 대한 사회적 인식: 뉴스기사와 트위터를 중심으로)

  • Lee, Sang-Soog;Yoo, Inhyeok;Kim, Jinhee
    • Journal of Digital Convergence
    • /
    • v.18 no.6
    • /
    • pp.9-16
    • /
    • 2020
  • The purpose of this study is to understand the public needs for AI education actively promoted and supported by the current government. In doing so, 11 metropolitan news articles and Twitter posts regarding AI education that have been posted from January 1, 2018 to December 31, 2019 were collected. Then, word frequency analysis using TF(Term Frequency) method and LDA(Latent Dirichlet Allocation) method of topic modeling analysis were conducted. The topics of the news articles turn out to be a macroscopic policy support such as 'training female manpower in the AI field' and 'curriculum reform of university and K-12', whereas the topics of twitter delineate more detailed social perception on future society, such as future competencies and pedagogical methods, including 'coexistence with intelligent robots', 'coding education', and 'humane education competence development'. The findings are expected to be used to suggest the implications for the composition and management of AI curriculum as well as the basic framework of human resources development in the future industry.

Design and Implementation of STEAM Game Contents for infant Learning Education using Gyroscope Sensor

  • Song, Mi-Young
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.1
    • /
    • pp.93-99
    • /
    • 2020
  • With the development of digital technology and the increasing demand for learning how to improve one's ability to solve problems through play and participation interactions, a variety of edutainment game contents are being developed. The edutainment game contents developed until recently have received a large number of contents for intelligence development and transfer of knowledge such as Korean and English mathematics for children and children. Recently, there have been various researches on the necessity and effect of STEAM education that foster convergent science and technology talents with comprehensive thinking ability and scientific inquiry spirit through the fusion education method among the subjects including science, technology, engineering, mathematics, And there is a growing need for the development of a parish suitable for STEAM education. However, there is a lack of STEAM educational content development that incorporates the technology of creative convergence talent training to develop talented people who can think and solve problems by crossing various academic boundaries. Therefore, this study develops game contents for early childhood education by combining STEAM education which foster convergent science and technology talents with comprehensive thinking ability and scientific inquiry spirit. And we designed and implemented STEAM game contents for infant learning education which can induce the interest of children and have fun by using gyroscope sensor of smartphone.

Detail Focused Image Classifier Model for Traditional Images (전통문화 이미지를 위한 세부 자질 주목형 이미지 자동 분석기)

  • Kim, Kuekyeng;Hur, Yuna;Kim, Gyeongmin;Yu, Wonhee;Lim, Heuiseok
    • Journal of the Korea Convergence Society
    • /
    • v.8 no.12
    • /
    • pp.85-92
    • /
    • 2017
  • As accessibility toward traditional cultural contents drops compared to its increase in production, the need for higher accessibility for continued management and research to exist. For this, this paper introduces an image classifier model for traditional images based on artificial neural networks, which converts the input image's features into a vector space and by utilizing a RNN based model it recognizes and compares the details of the input which enables the classification of traditional images. This enables the classifiers to classify similarly looking traditional images more precisely by focusing on the details. For the training of this model, a wide range of images were arranged and collected based on the format of the Korean information culture field, which contributes to other researches related to the fields of using traditional cultural images. Also, this research contributes to the further activation of demand, supply, and researches related to traditional culture.

Research on the Utilization of Recurrent Neural Networks for Automatic Generation of Korean Definitional Sentences of Technical Terms (기술 용어에 대한 한국어 정의 문장 자동 생성을 위한 순환 신경망 모델 활용 연구)

  • Choi, Garam;Kim, Han-Gook;Kim, Kwang-Hoon;Kim, You-eil;Choi, Sung-Pil
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.51 no.4
    • /
    • pp.99-120
    • /
    • 2017
  • In order to develop a semiautomatic support system that allows researchers concerned to efficiently analyze the technical trends for the ever-growing industry and market. This paper introduces a couple of Korean sentence generation models that can automatically generate definitional statements as well as descriptions of technical terms and concepts. The proposed models are based on a deep learning model called LSTM (Long Sort-Term Memory) capable of effectively labeling textual sequences by taking into account the contextual relations of each item in the sequences. Our models take technical terms as inputs and can generate a broad range of heterogeneous textual descriptions that explain the concept of the terms. In the experiments using large-scale training collections, we confirmed that more accurate and reasonable sentences can be generated by CHAR-CNN-LSTM model that is a word-based LSTM exploiting character embeddings based on convolutional neural networks (CNN). The results of this study can be a force for developing an extension model that can generate a set of sentences covering the same subjects, and furthermore, we can implement an artificial intelligence model that automatically creates technical literature.