• Title/Summary/Keyword: Machine Learning Framework

Search Result 229, Processing Time 0.027 seconds

Finding the best suited autoencoder for reducing model complexity

  • Ngoc, Kien Mai;Hwang, Myunggwon
    • Smart Media Journal
    • /
    • v.10 no.3
    • /
    • pp.9-22
    • /
    • 2021
  • Basically, machine learning models use input data to produce results. Sometimes, the input data is too complicated for the models to learn useful patterns. Therefore, feature engineering is a crucial data preprocessing step for constructing a proper feature set to improve the performance of such models. One of the most efficient methods for automating feature engineering is the autoencoder, which transforms the data from its original space into a latent space. However certain factors, including the datasets, the machine learning models, and the number of dimensions of the latent space (denoted by k), should be carefully considered when using the autoencoder. In this study, we design a framework to compare two data preprocessing approaches: with and without autoencoder and to observe the impact of these factors on autoencoder. We then conduct experiments using autoencoders with classifiers on popular datasets. The empirical results provide a perspective regarding the best suited autoencoder for these factors.

Compressive strength estimation of eco-friendly geopolymer concrete: Application of hybrid machine learning techniques

  • Xiang, Yang;Jiang, Daibo;Hateo, Gou
    • Steel and Composite Structures
    • /
    • v.45 no.6
    • /
    • pp.877-894
    • /
    • 2022
  • Geopolymer concrete (GPC) has emerged as a feasible choice for construction materials as a result of the environmental issues associated with the production of cement. The findings of this study contribute to the development of machine learning methods for estimating the properties of eco-friendly concrete to help reduce CO2 emissions in the construction industry. The compressive strength (fc) of GPC is predicted using artificial intelligence approaches in the present study when ground granulated blast-furnace slag (GGBS) is substituted with natural zeolite (NZ), silica fume (SF), and varying NaOH concentrations. For this purpose, two machine learning methods multi-layer perceptron (MLP) and radial basis function (RBF) were considered and hybridized with arithmetic optimization algorithm (AOA), and grey wolf optimization algorithm (GWO). According to the results, all methods performed very well in predicting the fc of GPC. The proposed AOA - MLP might be identified as the outperformed framework, although other methodologies (AOA - RBF, GWO - RBF, and GWO - MLP) were also reliable in the fc of GPC forecasting process.

Improving streamflow and flood predictions through computational simulations, machine learning and uncertainty quantification

  • Venkatesh Merwade;Siddharth Saksena;Pin-ChingLi;TaoHuang
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2023.05a
    • /
    • pp.29-29
    • /
    • 2023
  • To mitigate the damaging impacts of floods, accurate prediction of runoff, streamflow and flood inundation is needed. Conventional approach of simulating hydrology and hydraulics using loosely coupled models cannot capture the complex dynamics of surface and sub-surface processes. Additionally, the scarcity of data in ungauged basins and quality of data in gauged basins add uncertainty to model predictions, which need to be quantified. In this presentation, first the role of integrated modeling on creating accurate flood simulations and inundation maps will be presented with specific focus on urban environments. Next, the use of machine learning in producing streamflow predictions will be presented with specific focus on incorporating covariate shift and the application of theory guided machine learning. Finally, a framework to quantify the uncertainty in flood models using Hierarchical Bayesian Modeling Averaging will be presented. Overall, this presentation will highlight that creating accurate information on flood magnitude and extent requires innovation and advancement in different aspects related to hydrologic predictions.

  • PDF

SEQUENTIAL MINIMAL OPTIMIZATION WITH RANDOM FOREST ALGORITHM (SMORF) USING TWITTER CLASSIFICATION TECHNIQUES

  • J.Uma;K.Prabha
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.4
    • /
    • pp.116-122
    • /
    • 2023
  • Sentiment categorization technique be commonly isolated interested in threes significant classifications name Machine Learning Procedure (ML), Lexicon Based Method (LB) also finally, the Hybrid Method. In Machine Learning Methods (ML) utilizes phonetic highlights with apply notable ML algorithm. In this paper, in classification and identification be complete base under in optimizations technique called sequential minimal optimization with Random Forest algorithm (SMORF) for expanding the exhibition and proficiency of sentiment classification framework. The three existing classification algorithms are compared with proposed SMORF algorithm. Imitation result within experiential structure is Precisions (P), recalls (R), F-measures (F) and accuracy metric. The proposed sequential minimal optimization with Random Forest (SMORF) provides the great accuracy.

Modeling Framework for Continuous Dynamic Systems Using Machine Learning of Hypothetical Model (가설적 모델의 기계학습을 이용한 연속시간 동적시스템 모델링 프레임워크)

  • Hae Sang Song;Tag Gon Kim
    • Journal of the Korea Society for Simulation
    • /
    • v.32 no.1
    • /
    • pp.13-21
    • /
    • 2023
  • This paper proposes a method of automatically generating a model through a machine learning technique by setting a hypothetical model in the form of a gray box or black box with unknown parameters, when the big data of the actual system is given. We implements the proposed framework and conducts experiments to find an appropriate model among various hypothesis models and compares the cost and fitness of them. As a result we find that the proposed framework works well with continuous systems that could be modeled with ordinary differential equation. This technique is expected to be used well for the purpose of automatically updating the consistency of the digital twin model or predicting the output for new inputs using recently generated big data.

Development of a Framework for Improvement of Sensor Data Quality from Weather Buoys (해양기상부표의 센서 데이터 품질 향상을 위한 프레임워크 개발)

  • Ju-Yong Lee;Jae-Young Lee;Jiwoo Lee;Sangmun Shin;Jun-hyuk Jang;Jun-Hee Han
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.46 no.3
    • /
    • pp.186-197
    • /
    • 2023
  • In this study, we focus on the improvement of data quality transmitted from a weather buoy that guides a route of ships. The buoy has an Internet-of-Thing (IoT) including sensors to collect meteorological data and the buoy's status, and it also has a wireless communication device to send them to the central database in a ground control center and ships nearby. The time interval of data collected by the sensor is irregular, and fault data is often detected. Therefore, this study provides a framework to improve data quality using machine learning models. The normal data pattern is trained by machine learning models, and the trained models detect the fault data from the collected data set of the sensor and adjust them. For determining fault data, interquartile range (IQR) removes the value outside the outlier, and an NGBoost algorithm removes the data above the upper bound and below the lower bound. The removed data is interpolated using NGBoost or long-short term memory (LSTM) algorithm. The performance of the suggested process is evaluated by actual weather buoy data from Korea to improve the quality of 'AIR_TEMPERATURE' data by using other data from the same buoy. The performance of our proposed framework has been validated through computational experiments based on real-world data, confirming its suitability for practical applications in real-world scenarios.

Developing an Intrusion Detection Framework for High-Speed Big Data Networks: A Comprehensive Approach

  • Siddique, Kamran;Akhtar, Zahid;Khan, Muhammad Ashfaq;Jung, Yong-Hwan;Kim, Yangwoo
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.8
    • /
    • pp.4021-4037
    • /
    • 2018
  • In network intrusion detection research, two characteristics are generally considered vital to building efficient intrusion detection systems (IDSs): an optimal feature selection technique and robust classification schemes. However, the emergence of sophisticated network attacks and the advent of big data concepts in intrusion detection domains require two more significant aspects to be addressed: employing an appropriate big data computing framework and utilizing a contemporary dataset to deal with ongoing advancements. As such, we present a comprehensive approach to building an efficient IDS with the aim of strengthening academic anomaly detection research in real-world operational environments. The proposed system has the following four characteristics: (i) it performs optimal feature selection using information gain and branch-and-bound algorithms; (ii) it employs machine learning techniques for classification, namely, Logistic Regression, Naïve Bayes, and Random Forest; (iii) it introduces bulk synchronous parallel processing to handle the computational requirements of large-scale networks; and (iv) it utilizes a real-time contemporary dataset generated by the Information Security Centre of Excellence at the University of Brunswick (ISCX-UNB) to validate its efficacy. Experimental analysis shows the effectiveness of the proposed framework, which is able to achieve high accuracy, low computational cost, and reduced false alarms.

Understanding of the Overview of Quality 4.0 Using Text Mining (텍스트마이닝을 활용한 품질 4.0 연구동향 분석)

  • Kim, Minjun
    • Journal of Korean Society for Quality Management
    • /
    • v.51 no.3
    • /
    • pp.403-418
    • /
    • 2023
  • Purpose: The acceleration of technological innovation, specifically Industry 4.0, has triggered the emergence of a quality management paradigm known as Quality 4.0. This study aims to provide a systematic overview of dispersed studies on Quality 4.0 across various disciplines and to stimulate further academic discussions and industrial transformations. Methods: Text mining and machine learning approaches are applied to learn and identify key research topics, and the suggested key references are manually reviewed to develop a state-of-the-art overview of Quality 4.0. Results: 1) A total of 27 key research topics were identified based on the analysis of 1234 research papers related to Quality 4.0. 2) A relationship among the 27 key research topics was identified. 3) A multilevel framework consisting of technological enablers, business methods and strategies, goals, application industries of Quality 4.0 was developed. 4) The trends of key research topics was analyzed. Conclusion: The identification of 27 key research topics and the development of the Quality 4.0 framework contribute to a better understanding of Quality 4.0. This research lays the groundwork for future academic and industrial advancements in the field and encourages further discussions and transformations within the industry.

A machine learning framework for performance anomaly detection

  • Hasnain, Muhammad;Pasha, Muhammad Fermi;Ghani, Imran;Jeong, Seung Ryul;Ali, Aitizaz
    • Journal of Internet Computing and Services
    • /
    • v.23 no.2
    • /
    • pp.97-105
    • /
    • 2022
  • Web services show a rapid evolution and integration to meet the increased users' requirements. Thus, web services undergo updates and may have performance degradation due to undetected faults in the updated versions. Due to these faults, many performances and regression anomalies in web services may occur in real-world scenarios. This paper proposed applying the deep learning model and innovative explainable framework to detect performance and regression anomalies in web services. This study indicated that upper bound and lower bound values in performance metrics provide us with the simple means to detect the performance and regression anomalies in updated versions of web services. The explainable deep learning method enabled us to decide the precise use of deep learning to detect performance and anomalies in web services. The evaluation results of the proposed approach showed us the detection of unusual behavior of web service. The proposed approach is efficient and straightforward in detecting regression anomalies in web services compared with the existing approaches.

Language Matters: A Systemic Functional Linguistics-Enhanced Machine Learning Framework for Cyberbullying Detection

  • Raghad Altowairgi;Ala Eshamwi;Lobna Hsairi
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.9
    • /
    • pp.192-198
    • /
    • 2023
  • Cyberbullying is a growing problem among adolescents and can have serious psychological and emotional consequences for the victims. In recent years, machine learning techniques have emerged as promising approach for detecting instances of cyberbullying in online communication. This research paper focuses on developing a machine learning models that are able to detect cyberbullying including support vector machines, naïve bayes, and random forests. The study uses a dataset of real-world examples of cyberbullying collected from Twitter and extracts features that represents the ideational metafunction, then evaluates the performance of each algorithm before and after considering the theory of systemic functional linguistics in terms of precision, recall, and F1-score. The result indicates that all three algorithms are effective at detecting cyberbullying with 92% for naïve bayes and an accuracy of 93% for both SVM and random forests. However, the study also highlights the challenges of accurately detecting cyberbullying, particularly given the nuanced and context-dependent nature of online communication. This paper concludes by discussing the implications of these findings for future research and the development of practical tool for cyberbullying prevention and intervention.