• Title/Summary/Keyword: Missing

Search Result 2,817, Processing Time 0.025 seconds

한 인구학도의 회고

  • 김택일
    • Korea journal of population studies
    • /
    • v.11 no.1
    • /
    • pp.1-13
    • /
    • 1988
  • This study examines the sampling bias that may have resulted from the large number of missing observations. Despite well-designed and reliable sampling procedures, the observed sample values in DSFH(Demographic Survey on Changes in Family and Household Structure, Japan) included many missing observations. The head administerd survey method of DSFH resulted in a large number of missing observations regarding characteristics of elderly non-head parents and their children. In addition, the response probability of a particular item in DSFH significantly differs by characteristics of elderly parents and their children. Furthermore, missing observations of many items occurred simultaneously. This complex pattern of missing observations critically limits the ability to produce an unbiased analysis. First, the large number of missing observations is likely to cause a misleading estimate of the standard error. Even worse, the possible dependency of missing observations on their latent values is likely to produce biased estimates of covariates. Two models are employed to solve the possible inference biases. First, EM algorithm is used to infer the missing values based on the knowledge of the association between the observed values and other covariates. Second, a selection model was employed given the suspicion that the probability of missing observations of proximity depends on its unobserved outcome.

  • PDF

A data extension technique to handle incomplete data (불완전한 데이터를 처리하기 위한 데이터 확장기법)

  • Lee, Jong Chan
    • Journal of the Korea Convergence Society
    • /
    • v.12 no.2
    • /
    • pp.7-13
    • /
    • 2021
  • This paper introduces an algorithm that compensates for missing values after converting them into a format that can represent the probability for incomplete data including missing values in training data. In the previous method using this data conversion, incomplete data was processed by allocating missing values with an equal probability that missing variables can have. This method applied to many problems and obtained good results, but it was pointed out that there is a loss of information in that all information remaining in the missing variable is ignored and a new value is assigned. On the other hand, in the new proposed method, only complete information not including missing values is input into the well-known classification algorithm (C4.5), and the decision tree is constructed during learning. Then, the probability of the missing value is obtained from this decision tree and assigned as an estimated value of the missing variable. That is, some lost information is recovered using a lot of information that has not been lost from incomplete learning data.

Missing Value Imputation Technique for Water Quality Dataset

  • Jin-Young Jun;Youn-A Min
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.4
    • /
    • pp.39-46
    • /
    • 2024
  • Many researchers make efforts to evaluate water quality using various models. Such models require a dataset without missing values, but in real world, most datasets include missing values for various reasons. Simple deletion of samples having missing value(s) could distort distribution of the underlying data and pose a significant risk of biasing the model's inference when the missing mechanism is not MCAR. In this study, to explore the most appropriate technique for handing missing values in water quality data, several imputation techniques were experimented based on existing KNN and MICE imputation with/without the generative neural network model, Autoencoder(AE) and Denoising Autoencoder(DAE). The results shows that KNN and MICE combined imputation without generative networks provides the closest estimated values to the true values. When evaluating binary classification models based on support vector machine and ensemble algorithms after applying the combined imputation technique to the observed water quality dataset with missing values, it shows better performance in terms of Accuracy, F1 score, RoC-AuC score and MCC compared to those evaluated after deleting samples having missing values.

Imputation method for missing data based on measure of property (특성도를 이용한 결측치 대체방법)

  • Kim, Hyungju;Kim, Dongjae
    • The Korean Journal of Applied Statistics
    • /
    • v.30 no.3
    • /
    • pp.463-473
    • /
    • 2017
  • How to handle missing data is a main issue in clinical trials. We impute missing data based on missing data that follows a mechanism according to the intention-to-treat rule. However, using the right imputation method for missing data is very important because this supposition is unclear. We suggest a new imputation method for missing data using agreement and maintenance introduced by Kang and Kim (1997). We give an example and adapt a Monte Carlo simulation to compare the performance between the established method and the suggested method.

PhysioCover: Recovering the Missing Values in Physiological Data of Intensive Care Units

  • Kim, Sun-Hee;Yang, Hyung-Jeong;Kim, Soo-Hyung;Lee, Guee-Sang
    • International Journal of Contents
    • /
    • v.10 no.2
    • /
    • pp.47-58
    • /
    • 2014
  • Physiological signals provide important clues in the diagnosis and prediction of disease. Analyzing these signals is important in health and medicine. In particular, data preprocessing for physiological signal analysis is a vital issue because missing values, noise, and outliers may degrade the analysis performance. In this paper, we propose PhysioCover, a system that can recover missing values of physiological signals that were monitored in real time. PhysioCover integrates a gradual method and EM-based Principle Component Analysis (PCA). This approach can (1) more readily recover long- and short-term missing data than existing methods, such as traditional EM-based PCA, linear interpolation, 5-average and Missing Value Singular Value Decomposition (MSVD), (2) more effectively detect hidden variables than PCA and Independent component analysis (ICA), and (3) offer fast computation time through real-time processing. Experimental results with the physiological data of an intensive care unit show that the proposed method assigns more accurate missing values than previous methods.

Missing Modes in Fabry-Perot Laser Diodes (Fabry-Perot 레이저 다이오드의 Missing Mode)

  • Lee, Dong-Soo
    • Journal of the Korean Institute of Illuminating and Electrical Installation Engineers
    • /
    • v.19 no.1
    • /
    • pp.9-14
    • /
    • 2005
  • Mode missing of Fabry-Perot laser diodes has been modeled using the time domain laser model(TDLM). Fabry-Perot laser diodes that have structure of ripple in the waveguide of active layer or defects inside the active layer were simulated. For accurate simulation, the nonlinear effects were included such as spatial hole burning(SHB) and gain saturation. From the simulation results, it was founded that the defect inside the active layer in laser diodes has a strong influence on mode missing rather than the waveguide ripple. The simulation results are confirmed with the fabricated Fabry-Perot laser diodes by measuring the longitudinal mode spectra as a function of temperature from $25[^{\circ}C]\;to\;85[^{\circ}C]$.

Intelligent missing persons index system Implementation based on the OpenCV image processing and TensorFlow Deep-running Image Processing

  • Baek, Yeong-Tae;Lee, Se-Hoon;Kim, Ji-Seong
    • Journal of the Korea Society of Computer and Information
    • /
    • v.22 no.1
    • /
    • pp.15-21
    • /
    • 2017
  • In this paper, we present a solution to the problems caused by using only text - based information as an index element when a commercialized missing person indexing system indexes missing persons registered in the database. The existing system could not be used for the missing persons inquiry because it could not formalize the image of the missing person registered together when registering the missing person. To solve these problems, we propose a method to extract the similarity of images by using OpenCV image processing and TensorFlow deep - running image processing, and to process images of missing persons to process them into meaningful information. In order to verify the indexing method used in this paper, we constructed a Web server that operates to provide the information that is most likely to be needed to users first, using the image provided in the non - regular environment of the same subject as the search element.

A CLINICAL STUDY ON THE CONGENITALLY MISSING TEETH IN MESIODENS CASES (상악 정중 과잉치 증례 중 선천 결손치 발생에 관한 임상적 고찰)

  • Kwon, Min-Seok;Jung, Tae-Sung;Kim, Shin
    • Journal of the korean academy of Pediatric Dentistry
    • /
    • v.29 no.4
    • /
    • pp.574-578
    • /
    • 2002
  • Mesiodens is developmental tooth anomaly which is commonly found in clinical pediatric dentistry. however, it may cause many partial problem in tooth alignment when congenitally missing teeth was accompanied by mesiodens. The terms, concomitant hypodontia and hyperdontia' and oligo-pleiodontia' have been used to describe the condition in witch developmental absence of teeth and supernumerary teeth are present in the same individual. Only a few case reports of this rare condition which is opposite developmental phenomena exist in the literature. The purpose of this study is survey of congenitally missing teeth in mesiodens case and to compare previous literature of congenitally missing teeth in normal. The subjects were 310 children(247 male and 63 female) at the age from 5 to 12 years visiting the Department of Pediatric Dentistry, Pusan National University Hospital with mesiodens for last 3 years. With their pantomograms we studied congenitally missing teeth except permanent 3rd molar. 1. The preference of congenitally missing teeth in mesiodens cases was revealed to be 17.1%(53 out of 310 in total), and there was a higher prevalence in females(22.2%) than in males(15.8%). 2. The most frequently missing teeth were maxillary lateral incisors(22.7%) and mandibular second premolars(22.7%), followed by maxillary second premolar(17.3%), and mandibular lateral incisors(16.0%). There was no significant differences between maxilla(49.3%) and mandible(50.7%). 3. In number of congenitally missing teeth per person, 69.9% had one missing tooth, 22.7% had two missing teeth and 9.4% had three missing teeth.

  • PDF

Japanese Private Detective Investigation of Long Term Missing Person (일본 탐정의 장기미제 실종자 조사)

  • Shin, Jae-Hun;Kim, Sang-Woon
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.7
    • /
    • pp.412-420
    • /
    • 2019
  • The purpose of this study is to suggest legalization of private investigation system by giving an example of Japanese detective investigation on people who have been missing for a long time. To accomplish the research purpose, this study reviewed literature and used preceding researches on Japanese detective system to complete theoretical background. This study also identified the missing people in Korea and Japan to accomplish the research purpose. Conclusion: In Japan, about 80,000 people are reported to be missing every year. Although most of them are found on that day or within a week, some of them are not found for more than a week. There even a case where the person was missing for more than 3 years. In such case, the Japanese citizens requests detectives to find the missing person instead of depending on the police. When Japanese detectives are paid by their client, they provide the security service requested by the client. Japanese detectives receive about 100,000 yen to 700,000 yen for finding missing person and they find the missing person through investigation on voluntary or involuntary missing person. Such activities of Japanese detectives point out the necessity of introducing private investigation system in Korea. Currently, most missing people are not found in Korea. Introduction of private investigation system will help in finding the missing person, reducing the excessive workload of police, and creating jobs.

Nonstationary Time Series and Missing Data

  • Shin, Dong-Wan;Lee, Oe-Sook
    • The Korean Journal of Applied Statistics
    • /
    • v.23 no.1
    • /
    • pp.73-79
    • /
    • 2010
  • Missing values for unit root processes are imputed by the most recent observations. Treating the imputed observations as if they are complete ones, semiparametric unit root tests are extended to missing value situations. Also, an invariance principle for the partial sum process of the imputed observations is established under some mild conditions, which shows that the extended tests have the same limiting null distributions as those based on complete observations. The proposed tests are illustrated by analyzing an unequally spaced real data set.