• Title/Summary/Keyword: Data Cleaning

Search Result 424, Processing Time 0.024 seconds

An Intelligent System for Filling of Missing Values in Weather Data

  • Maqsood Ali Solangi;Ghulam Ali Mallah;Shagufta Naz;Jamil Ahmed Chandio;Muhammad Bux Soomro
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.9
    • /
    • pp.95-99
    • /
    • 2023
  • Recently Machine Learning has been considered as one of the active research areas of Computer Science. The various Artificial Intelligence techniques are used to solve the classification problems of environmental sciences, biological sciences, and medical sciences etc. Due to the heterogynous and malfunctioning weather sensors a considerable amount of noisy data with missing is generated, which is alarming situation for weather prediction stockholders. Filling of these missing values with proper method is really one of the significant problems. The data must be cleaned before applying prediction model to collect more precise & accurate results. In order to solve all above stated problems, this research proposes a novel weather forecasting system which consists upon two steps. The first step will prepare data by reducing the noise; whereas a decision model is constructed at second step using regression algorithm. The Confusion Matrix will be used to evaluation the proposed classifier.

The Effects of Preferred Job Type of University Students on the Confirmation Bias and Job Anxiety (대학생의 선호직업유형이 확증편향과 취업불안에 미치는 영향)

  • Roh, Seon-Hee;Kim, Ki-Seung
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.20 no.8
    • /
    • pp.190-199
    • /
    • 2019
  • This quantitative study analyzed the influence of college students' preferred type of occupation on a confirmation bias and job anxiety during the process of making a career decision. The questionnaires were distributed to university students in Seoul and the metropolitan area for 500 weeks from July 10 2017 to August 8, 2017. Among them, 482 valid samples of data were analyzed by data coding and data cleaning usin SPSS 18.0 statistics and the AMOS 18.0 program. The main results of this study are that the type of business preference for an affirmative bias has a positive (+) direct influence (${\beta}=.374$) and the type of freedom has a positive direct influence (${\beta}=.326$) and a negative direct influence (${\beta}=-.274$). In the case of job anxiety, the influence of job type is more increased. The confirmation bias shows that the business type and freestyle type find cause in effort or achievement motive, while rect type is recognized as social environment and structural problem. In conclusion, there is a difference in the degree of confirmation bias and job insecurity. This study shows that college students' preferred occupation types can help them to understand the bias and anxiety that they have in preparing for the job and help to reduce job anxiety, and these findings are expected to be useful for career guidance.

A Smoothing Data Cleaning based on Adaptive Window Sliding for Intelligent RFID Middleware Systems (지능적인 RFID 미들웨어 시스템을 위한 적응형 윈도우 슬라이딩 기반의 유연한 데이터 정제)

  • Shin, DongCheon;Oh, Dongok;Ryu, SeungWan;Park, Seikwon
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.3
    • /
    • pp.1-18
    • /
    • 2014
  • Over the past years RFID/SN has been an elementary technology in a diversity of applications for the ubiquitous environments, especially for Internet of Things. However, one of obstacles for widespread deployment of RFID technology is the inherent unreliability of the RFID data streams by tag readers. In particular, the problem of false readings such as lost readings and mistaken readings needs to be treated by RFID middleware systems because false readings ultimately degrade the quality of application services due to the dirty data delivered by middleware systems. As a result, for the higher quality of services, an RFID middleware system is responsible for intelligently dealing with false readings for the delivery of clean data to the applications in accordance with the tag reading environment. One of popular techniques used to compensate false readings is a sliding window filter. In a sliding window scheme, it is evident that determining optimal window size intelligently is a nontrivial important task in RFID middleware systems in order to reduce false readings, especially in mobile environments. In this paper, for the purpose of reducing false readings by intelligent window adaption, we propose a new adaptive RFID data cleaning scheme based on window sliding for a single tag. Unlike previous works based on a binomial sampling model, we introduce the weight averaging. Our insight starts from the need to differentiate the past readings and the current readings, since the more recent readings may indicate the more accurate tag transitions. Owing to weight averaging, our scheme is expected to dynamically adapt the window size in an efficient manner even for non-homogeneous reading patterns in mobile environments. In addition, we analyze reading patterns in the window and effects of decreased window so that a more accurate and efficient decision on window adaption can be made. With our scheme, we can expect to obtain the ultimate goal that RFID middleware systems can provide applications with more clean data so that they can ensure high quality of intended services.

Learning Context Awareness Model based on User Feedback for Smart Home Service

  • Kwon, Seongcheol;Kim, Seyoung;Ryu, Kwang Ryel
    • Journal of the Korea Society of Computer and Information
    • /
    • v.22 no.7
    • /
    • pp.17-29
    • /
    • 2017
  • IRecently, researches on the recognition of indoor user situations through various sensors in a smart home environment are under way. In this paper, the case study was conducted to determine the operation of the robot vacuum cleaner by inferring the user 's indoor situation through the operation of home appliances, because the indoor situation greatly affects the operation of home appliances. In order to collect learning data for indoor situation awareness model learning, we received feedbacks from user when there was a mistake about the cleaning situation. In this paper, we propose a semi-supervised learning method using user feedback data. When we receive a user feedback, we search for the labels of unlabeled data that most fit the feedbacks collected through genetic algorithm, and use this data to learn the model. In order to verify the performance of the proposed algorithm, we performed a comparison experiments with other learning algorithms in the same environment and confirmed that the performance of the proposed algorithm is better than the other algorithms.

Predicting idiopathic pulmonary fibrosis (IPF) disease in patients using machine approaches

  • Ali, Sikandar;Hussain, Ali;Kim, Hee-Cheol
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2021.05a
    • /
    • pp.144-146
    • /
    • 2021
  • Idiopathic pulmonary fibrosis (IPF) is one of the most dreadful lung diseases which effects the performance of the lung unpredictably. There is no any authentic natural history discovered yet pertaining to this disease and it has been very difficult for the physicians to diagnosis this disease. With the advent of Artificial intelligent and its related technologies this task has become a little bit easier. The aim of this paper is to develop and to explore the machine learning models for the prediction and diagnosis of this mysterious disease. For our study, we got IPF dataset from Haeundae Paik hospital consisting of 2425 patients. This dataset consists of 502 features. We applied different data preprocessing techniques for data cleaning while making the data fit for the machine learning implementation. After the preprocessing of the data, 18 features were selected for the experiment. In our experiment, we used different machine learning classifiers i.e., Multilayer perceptron (MLP), Support vector machine (SVM), and Random forest (RF). we compared the performance of each classifier. The experimental results showed that MLP outperformed all other compared models with 91.24% accuracy.

  • PDF

Implementation of a File System for Flash Memory (플래시 메모리를 위한 파일 시스템의 구현)

  • Park, Sang-Ho;Ahn, Woo-Hyun;Park, Dae-Yeon;Kim, Jeong-Ki;Park, Sung-Min
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.7 no.5
    • /
    • pp.402-415
    • /
    • 2001
  • Advantages of flash memories are their shock resistance and fast read speed, which is much faster than that of a HDD. Because of these characteristics, they are increasingly used in the traditional household electric appliance and portable handset and therefore, development of file systems which use them as storage medium is increasingly needed. But they have two problems as storage medium. First, data stored in them cannot be overwritten: it must be erased before new data can be stored. Unfortunately, this erase operation usually takes about one second. Consequently, updating data in flash memories takes long time. In this paper, their problem is solved by using a data update mechanism like LFS(Log-structured File System). Second, their erase operations are restricted. We propose novel cleaning policy in order to increase the life cycle. We implemented FAT file system, which is suitable to small storage medium and solved problems, which usually happen in implementing FAT. We evaluated the performance of sequential writes and random writes on our implemented flash file system.

  • PDF

The effect of cleaning methods on bond strength of zirconia after saliva contamination (타액으로 오염된 지르코니아 수복물의 접착강도에 세척 방법들이 미치는 영향)

  • Shim, Young-Bo;Choi, An-Na;Son, Sung-Ae;Jung, Kyoung-Hwa;Kwon, Yong Hoon;Park, Jeong-Kil
    • Korean Journal of Dental Materials
    • /
    • v.44 no.1
    • /
    • pp.61-68
    • /
    • 2017
  • This study evaluated the effects of various cleaning methods on the shear bond strength of zirconia ceramics after saliva contamination. Eighty zirconia disk specimens were divided into 8 groups. All groups were treated with one coat of MDP primer. All specimens (except the negative control) were contaminated with human saliva on the zirconia surface. The positive control went through the bonding procedure immediately after contamination without any cleaning procedure. With the exception of control groups, the remaining six groups were rinsed with water and either applied with MDP recoating (WATER+MDP) or without MDP recoating (WATER). While some were cleaned with a Ivoclean with MDP recoating (IVOCLEAN+MDP) or not applied with MDP recoating(IVOCLEAN), others were cleaned with a 1% NaOCl solution with MDP recoating (NaOCl+MDP) or without MDP recoating (NaOCl). The shear bond strength of all specimens were measured after being stored in distilled water at $37^{\circ}C$ for 24 hours. The data was analyzed statistically by an analysis of ANOVA, Tukey's post hoc test and Student's t-test was used to compare the shear bond strength according to the re-coating of MDP after the cleaning procedure. The positive control group showed the lowest shear bond strength value, and the WATER group and NaOCl group showed no significant difference when compared to the positive control group. The IVOCLEAN group showed significantly higher shear bond strength when compared to Water group and NaOCl group but not with the group of negative control. After rinsing with water or the NaOCl solution, the comparison of the single coating of MDP and re-coating of MDP showed different shear bond strengths but there was no significant difference to the negative control. After rinsing with Ivoclean, there was no significant difference to the negative control regardless of the recoating of MDP. In conclusion, the shear bond strength was affected by the cleansing procedure and Ivoclean was found to be effective regardless of the re-coating of MDP. When water or the NaOCl solution is used to remove surface contaminants, the re-coating of MDP provides a positive effect on cementation.

A Study of Foodwaste Treatment Technology Using Eatrhworm (지렁이를 이용한 음식물쓰레기 처리에 관한 연구)

  • Lee, Chang-Ho;Lee, Jong-Min;Bae, Sung-Geun;Jeon, Sung-Kyun;Kim, Jong-Oh
    • Journal of the Korea Organic Resources Recycling Association
    • /
    • v.13 no.1
    • /
    • pp.71-78
    • /
    • 2005
  • This study was performed to obtain the basic data on current situations and challengeable aspects of vermicomposting industries in Korea and to develope foodwaste treatment technology using earthworm. Pre-treatment process of food waste consists of cleaning, grinding, mixing and others, and salinity of food waste was 0.7% on average, thus not suitable for feed of worms. When applying wash water and going through the cleaning process, however, salinity was dropped to 0.38%, indicating that cleaning process applying wash water is necessary to control the proper salinity for growth of worms. In the food waste processing experiment applying worms, it has been presented that smaller particles induce higher processing efficiency. As a result, food waste processing by worms is possible through maintaining the suitable environment for survival of worms from input process to generation of cast. Cast generated after processing food waste was proved to be most suitable for product standard of compost thus cast is presented to be recycled after processing the food waste applying the worms.

  • PDF

Awareness and using status on long-term care insurance and insurance benefits (노인 장기요양보험 이용실태 및 서비스내용에 대한 인식)

  • Jung, Jae-Yeon;Kim, Soo-Hwa;Kim, Young-Kyung;Ahn, Se-Youn;Yoo, Eun-Mi;Choi, Boo-Keun;Hwang, Yoon-Sook;Han, Su-Jin
    • Journal of Korean society of Dental Hygiene
    • /
    • v.16 no.3
    • /
    • pp.373-381
    • /
    • 2016
  • Objectives: The purpose of the study is to investigate the awareness toward use and service contents of long term care for the elderly. Methods: A self-reported questionnaire was completed by 296 adults from August 13 to October 20, 2014. The questionnaire consisted of general characteristics of the subjects, awareness toward long term care insurance for the elderly, awareness toward long term oral health care services, use of long term care service and use intention for the long term care insurance. Data were analyzed by SPSS 18.0 program. Results: Those who were aware of the long term care insurance accounted for 55.4 percent. Approximately 50 percent of the respondents recognized long term care service items, home visit care, home visit bathing, and home visit nursing. Most of the respondents had information of long term care services by way of mass media and direct contact. Only 13.4 percent of the respondents were aware of the oral health service in the long term care insurance. The subjects were aware of denture cleaning, oral cleaning and oral health education out of oral health service in order; and oral health services that needed to be offered were denture cleaning, oral health education and professional toothbrushing. They reported that dental hygienists were the most important manpower that offered the efficient oral health care services. They answered that professional manpower and financial support are required for oral health services. The positive thinking to long term care insurance accounted for 89.2 percent and 91.3 percent had use intention for oral health services. Conclusions: Many elderly people have mastication or dysphagic problems due to systemic diseases. Therefore, it is necessary to announce the long term care insurance and long term care services for the elderly people.

The Amount of House Mite Allergens & Dusts According to Environmental Factors of Patients With Allergic Rhinitis (알레르기성비염 환자 침실의 환경적 특성에 따른 집먼지 진드기 항원량과 먼지량)

  • Moon Jung Soon;Choi Soon Ock
    • Journal of Korean Public Health Nursing
    • /
    • v.10 no.2
    • /
    • pp.51-60
    • /
    • 1996
  • This study was conducted to provide basic knowledges of effective environmental control of patients with allergic rhinitis. From July to October 1995, 58 dust samples Were drawn from the bedrooms of patients with allergic rhinitis who were registerd at an allergy clinic of a hospital. Those samples were examined for the amount of house mite allergens & dusts. The data were analysed by using ANOVA and Pearson correiation coefficients. The results were as follows : 1. As for the amount of house mite allergens in terms of environmental factors, the amount of house mite allergens of using washing water temperature of bedding above than $55^{\circ}C$ was significantly lower than that of below $54^{\circ}C$. Other environmental factors such as type of house, area of bed room floor, bed in bedroom, bedroom cleaning by vacuum cleaner, days after bedding washing, relative humidity of bedroom were relate to the amount of house mite allergens. 2. As for the amount of dusts in terms of environmental factors, the amount of dusts of days of days after bedding washing more than 15 days was significantly lower than thant of less than 15days. Other environmental factors such as type of house, area of bedroom floor, bed in bedroom, bedroom cleaning by vacuum cleaner, was hing water taemperature of bedding, relative humidity of bedroom were not relate to the amount of dusts. 3. There was a significant positive correlation between the amount of house mite allergens and the amount of dusts. It may be conclusively said. the amount of dusts and house mite allergens were closely associated with the washing temperature and days after washing of bedding. Hence. intensive instruction for the methods of bedding washing was needs of the patients with allergic rhinitis.

  • PDF