• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.23 no.5
• /
• pp.495-507
• /
• 2019

Data sparsity is one of the main challenges for the recommender system. The recommender system contains massive data in which only a small part is the observed data and the others are missing data. Most studies assume that missing data is randomly missing from the dataset. Therefore, they only use observed data to train recommendation model, then recommend items to users. In actual case, however, missing data do not lost randomly. In our research, treat these missing data as negative examples of users' interest. Three sample methods are seamlessly integrated into SVD++ algorithm and then propose SVD++_W, SVD++_R and SVD++_KNN algorithm. Experimental results show that proposed sample methods effectively improve the precision in Top-N recommendation over the baseline algorithms. Among the three improved algorithms, SVD++_KNN has the best performance, which shows that the KNN sample method is a more effective way to extract the negative examples of the users' interest.

• Proceedings of the IEEK Conference
• /
• 2009.05a
• /
• pp.344-346
• /
• 2009

Cell type packaging line is more suitable for the products with various models and small quantities like mobile phone or mp3 player than conveyor type packaging line. Cell type packaging line is applicable to package various product models, but it can cause wrong product compositions and missing of items. So, automatic missing item detection system is needed. We designed an missing item detection system with a bar code reader, infrared sensors, and s digital camera. and also developed the programs for sensor data acquisition, image data processing, GUI, and data management.

• International Journal of Contents
• /
• v.10 no.2
• /
• pp.47-58
• /
• 2014

Physiological signals provide important clues in the diagnosis and prediction of disease. Analyzing these signals is important in health and medicine. In particular, data preprocessing for physiological signal analysis is a vital issue because missing values, noise, and outliers may degrade the analysis performance. In this paper, we propose PhysioCover, a system that can recover missing values of physiological signals that were monitored in real time. PhysioCover integrates a gradual method and EM-based Principle Component Analysis (PCA). This approach can (1) more readily recover long- and short-term missing data than existing methods, such as traditional EM-based PCA, linear interpolation, 5-average and Missing Value Singular Value Decomposition (MSVD), (2) more effectively detect hidden variables than PCA and Independent component analysis (ICA), and (3) offer fast computation time through real-time processing. Experimental results with the physiological data of an intensive care unit show that the proposed method assigns more accurate missing values than previous methods.

• Journal of the Korea Convergence Society
• /
• v.11 no.3
• /
• pp.1-7
• /
• 2020

This paper introduces two different techniques for dealing with incomplete data and algorithms for learning this data. The first method is to process the incomplete data by assigning the missing value with equal probability that the missing variable can have, and learn this data with the SVM. This technique ensures that the higher the frequency of missing for any variable, the higher the entropy so that it is not selected in the decision tree. This method is characterized by ignoring all remaining information in the missing variable and assigning a new value. On the other hand, the new method is to calculate the entropy probability from the remaining information except the missing value and use it as an estimate of the missing variable. In other words, using a lot of information that is not lost from incomplete learning data to recover some missing information and learn using deep learning. These two methods measure performance by selecting one variable in turn from the training data and iteratively comparing the results of different measurements with varying proportions of data lost in the variable.

• International conference on construction engineering and project management
• /
• 2024.07a
• /
• pp.271-276
• /
• 2024

The success of construction projects is influenced by various factors, with accurate management and prediction of the construction period playing a crucial role. The construction period is determined through contracts between the client and the contractor, and it is considered a key element in the management of construction projects, alongside cost management. To ensure the successful completion of projects, accurate prediction of the construction period is essential, as it aids in the efficient allocation of time and resources. The main objective of this study is to maximize the performance of construction period prediction models by applying and comparing various methods for handling missing data. Optimizing the model's performance requires accuracy and completeness of data, with the process of outlier removal and missing data imputation potentially having a significant impact on the model's predictive capability. During this process, the effect of changes in the dataset on model performance will be closely examined to identify the most effective method for handling missing data. Outlier removal and missing data imputation are crucial steps in the data preprocessing phase, and they can significantly improve the model's accuracy and reliability. This research aims to apply these data preprocessing methods and analyze their outcomes to find the most effective missing data imputation method for construction period prediction. After the selection process, considering the model's performance and stability, the mode imputation method was identified as the most suitable for predicting the construction period. The findings of this research are expected to contribute not only to improving the accuracy of construction period predictions but also to enhancing the overall efficiency and success rate of construction project management.

• Journal of the Korean Data and Information Science Society
• /
• v.17 no.3
• /
• pp.871-888
• /
• 2006

Many studies have been done to develop procedures to deal with missing values. Most common method is to reassign the other values to the missing data. The purpose of our study is to suggest adjusted Cohen methods and to compare the efficiency of them with other methods through a simulation study. The adjusted Cohen methods use an auxiliary variable to arrange ranking of the variable with missing values. It leads to a reduced mean square error(MSE) compared with the Cohen method.

• Communications for Statistical Applications and Methods
• /
• v.26 no.4
• /
• pp.411-430
• /
• 2019

Tumor development is driven by complex combinations of biological elements. Recent advances suggest that molecularly distinct subtypes of breast cancers may respond differently to pathway-targeted therapies. Thus, it is important to dissect pathway disturbances by integrating multiple molecular profiles, such as genetic, genomic and epigenomic data. However, missing data are often present in the -omic profiles of interest. Motivated by genomic data integration and imputation, we present a new statistical framework for pathway significance analysis. Specifically, we develop a new strategy for imputation of missing data in large-scale genomic studies, which adapts low-rank, structured matrix completion. Our iterative strategy enables us to impute missing data in complex configurations across multiple data platforms. In turn, we perform large-scale pathway analysis integrating gene expression, copy number, and methylation data. The advantages of the proposed statistical framework are demonstrated through simulations and real applications to breast cancer subtypes. We demonstrate superior power to identify pathway disturbances, compared with other imputation strategies. We also identify differential pathway activity across different breast tumor subtypes.

• The Korean Journal of Applied Statistics
• /
• v.25 no.2
• /
• pp.279-291
• /
• 2012

When fitting a Cox proportional hazards model with missing covariates, it is inefficient to exclude observations with missing values in the analysis. Furthermore, if the missing-data mechanism is not Missing Completely At Random(MCAR), it may lead to biased parameter estimation. Many approaches have been suggested to handle the Cox proportional hazards model when covariates are sometimes missing, but they are based on the selection model. This paper suggest an approach to handle Cox proportional hazards model with missing covariates by using the pattern-mixture model (Little, 1993). The pattern-mixture model is expressed by the joint distribution of survival time and the missing-data mechanism. In the pattern-mixture model, many models can be considered by setting up various restrictions, and different results under various restrictions indicate the sensitivity of the model due to missing covariates. A simulation study was conducted to show the sensitivity of parameter estimation under different restrictions in a pattern-mixture model. The proposed approach was also applied to mouse leukemia data.

• Journal of Preventive Medicine and Public Health
• /
• v.31 no.4 s.63
• /
• pp.875-884
• /
• 1998

Missing observations are common in medical research and health survey research. Several statistical methods to handle the missing data problem have been proposed. The EM algorithm (Expectation-Maximization algorithm) is one of the ways of efficiently handling the missing data problem based on sufficient statistics. In this paper, we developed statistical models and methods for survey data with multivariate missing observations. Especially, we adopted the EM algorithm to handle the multivariate missing observations. We assume that the multivariate observations follow a multivariate normal distribution, where the mean vector and the covariance matrix are primarily of interest. We applied the proposed statistical method to analyze data from a health survey. The data set we used came from a physician survey on Resource-Based Relative Value Scale(RBRVS). In addition to the EM algorithm, we applied the complete case analysis, which uses only completely observed cases, and the available case analysis, which utilizes all available information. The residual and normal probability plots were evaluated to access the assumption of normality. We found that the residual sum of squares from the EM algorithm was smaller than those of the complete-case and the available-case analyses.

• Journal of the Korean Statistical Society
• /
• v.26 no.3
• /
• pp.319-332
• /
• 1997

The application of multivariate linear rank statistics to data with item nonresponse is considered. Only a modest extension of the complete data techniques is required when the missing data may be thought of as a random sample, and an appropriate modification of the covariances is derived. A proof of the asymptotic multivariate normality is given. A review of some related results in the literature is presented and applications including longitudinal and repeated measures designs are discussed.