• Proceedings of the KSME Conference
• /
• 2008.11a
• /
• pp.1169-1175
• /
• 2008

The validation technique is classified with two methods whether to demand of additional experimental points. The method which requires additional experimental points such as RSME is actually impossible in engineering field. Therefore, the method which only use experimented points such as the cross validation technique is only available. But the cross validation not only requires considerable computational costs for generating metamodel each iterations, but also cannot measure quantitatively the fidelity of metamodel. In this research we propose a new validation technique for representative metamodels using an variance of metamodel and confidence interval information. The proposed validation technique computes confidence intervals using a variance information from the metamodel. This technique will have influence on choosing the accurate metamodel, constructing ensemble of each metamodels and advancing effectively sequential sampling technique.

• Transactions of the Korean Society of Mechanical Engineers A
• /
• v.34 no.5
• /
• pp.541-547
• /
• 2010

The rigorous validation of the accuracy of metamodels is an important topic in research on metamodel techniques. Although a leave-k-out cross-validation technique involves a considerably high computational cost, it cannot be used to measure the fidelity of metamodels. Recently, the mean$_0$ validation technique has been proposed to quantitatively determine the accuracy of metamodels. However, the use of mean$_0$ validation criterion may lead to premature termination of a sampling process even if the kriging model is inaccurate. In this study, we propose a new validation technique based on the mean and variance of the response evaluated when sequential sampling method, such as maximum entropy sampling, is used. The proposed validation technique is more efficient and accurate than the leave-k-out cross-validation technique, because instead of performing numerical integration, the kriging model is explicitly integrated to accurately evaluate the mean and variance of the response evaluated. The error in the proposed validation technique resembles a root mean squared error, thus it can be used to determine a stop criterion for sequential sampling of metamodels.

• Wind and Structures
• /
• v.18 no.4
• /
• pp.375-389
• /
• 2014

The high-frequency force-balance (HFFB) technique and its subsequent improvements are reviewed in this paper, including a discussion about nonlinear mode shape corrections, multi-force balance measurements, and using HFFB model to identify aeroelastic parameters. To apply the HFFB technique in engineering practice, various validation studies have been conducted. This paper presents the results from an analytical validation study for a simple building with nonlinear mode shapes, three experimental validation studies for more complicated buildings, and a field measurement comparison for a super-tall building in Hong Kong. The results of these validations confirm that the improved HFFB technique is generally adequate for engineering applications. Some technical limitations of HFFB are also discussed in this paper, especially for higher-order mode response that could be considerable for super tall buildings.

• Proceedings of the IEEK Conference
• /
• 2002.07b
• /
• pp.920-923
• /
• 2002

In this paper, we propose Class Diagram With Constraints (CDWC) as an object oriented modeling technique which makes validation possible in software development. CDWC is a simple and basic model for the object oriented analysis, and has a reasonable strictness for software developers. CDWC consists of class diagrams and constraints (invariant and pre/post conditions), using UML and a subset of OCL.. We introduce a method of validation of CDWC using the verification technique of algebraic formal specification language CafeOBJ.

• Proceedings of the Korean Statistical Society Conference
• /
• 2004.11a
• /
• pp.195-198
• /
• 2004

One Important problem in a cancer microarray study is to identify a set of genes from which a molecular prognostic indicator can be developed. In parallel with this problem is to validate the chosen set of genes. We develop in this note a K-fold cross validation procedure by combining a 'pre-validation' technique and a bootstrap resampling procedure in the Cox regression . The pre-validation technique predicts the microarray predictor of a case without having seen the true class level of the case. It was suggested by Tibshirani and Efron (2002) to avoid the possible over-fitting in the regression in which a microarray based predictor is employed. The bootstrap resampling procedure for the Cox regression was proposed by Sauerbrei and Schumacher (1992) as a means of overcoming the instability of a stepwise selection procedure. We apply this K-fold cross validation to the microarray data of 92 gastric cancers of which the experiment was conducted at Cancer Metastasis Research Center, Yonsei University. We also share some of our experience on the 'false positive' result due to the information leak.

• Journal of Biomedical Engineering Research
• /
• v.40 no.2
• /
• pp.48-54
• /
• 2019

This paper proposed a method of validation data augmentation for improving the grading accuracy of diabetic macular edema (DME) using deep learning. The data augmentation technique is basically applied in order to secure diversity of data by transforming one image to several images through random translation, rotation, scaling and reflection in preparation of input data of the deep neural network (DNN). In this paper, we apply this technique in the validation process of the trained DNN, and improve the grading accuracy by combining the classification results of the augmented images. To verify the effectiveness, 1,200 retinal images of Messidor dataset was divided into training and validation data at the ratio 7:3. By applying random augmentation to 359 validation data, $1.61{\pm}0.55%$ accuracy improvement was achieved in the case of six times augmentation (N=6). This simple method has shown that the accuracy can be improved in the N range from 2 to 6 with the correlation coefficient of 0.5667. Therefore, it is expected to help improve the diagnostic accuracy of DME with the grading information provided by the proposed DNN.

• Transactions of the Korean Society of Mechanical Engineers A
• /
• v.31 no.1 s.256
• /
• pp.55-61
• /
• 2007

Recently simulation model becomes an essential tool for analysis and design of a system but it is often expensive and time consuming as it becomes complicate to achieve reliable results. Therefore, high-fidelity simulation model needs to be replaced by an approximate model, the so-called metamodel. Metamodeling techniques include 3 components of sampling, metamodel and validation. Cross-validation approach has been proposed to provide sequnatially new sample point based on cross-validation error but it is very expensive because cross-validation must be evaluated at each stage. To enhance the cross-validation of metamodel, sequential sampling method using candidate points and representative cross-validation is proposed in this paper. The candidate and representative cross-validation approach of sequential sampling is illustrated for two-dimensional domain. To verify the performance of the suggested sampling technique, we compare the accuracy of the metamodels for various mathematical functions with that obtained by conventional sequential sampling strategies such as maximum distance, mean squared error, and maximum entropy sequential samplings. Through this research we team that the proposed approach is computationally inexpensive and provides good prediction performance.

• Transactions of the Korean Society of Mechanical Engineers A
• /
• v.36 no.8
• /
• pp.873-879
• /
• 2012

Metamodels have been developed with a variety of design optimization techniques in the field of structural engineering over the last decade because they are efficient, show excellent prediction performance, and provide easy interconnections into design frameworks. To construct a metamodel, a sequential procedure involving steps such as the design of experiments, metamodeling techniques, and validation techniques is performed. Because validation techniques can measure the accuracy of the metamodel, the number of presampled points for an accurate kriging metamodel is decided by the validation technique in the sequential kriging metamodel. Because the interpolation model such as the kriging metamodel based on computer experiments passes through responses at presampled points, additional analyses or reconstructions of the metamodels are required to measure the accuracy of the metamodel if existing validation techniques are applied. In this study, we suggest a sensitivity validation that does not require additional analyses or reconstructions of the metamodels. Fourteen two-dimensional mathematical problems and an engineering problem are illustrated to show the feasibility of the suggested method.

• Journal of the Korea Society for Simulation
• /
• v.18 no.4
• /
• pp.185-195
• /
• 2009

As most systems get more complicated, system analysis using simulation has been taken notice of. One of the core parts of simulation analysis is validation of a simulation model, and we can identify how well the simulation model represents the real system with this validation process. The difference between input data of two systems has an effect on the comparison between a simulation model and a real system at validation stage, and the result with such difference is not enough to ensure high credibility of the model. Accordingly, in this paper, we construct a model based on Trace-driven simulation which uses identical input data with the real system. On the other hand, to validate a model by each class, not by an unique statistic, we validate the model using a metric transformed from F-measure which estimates performance of a classifier in data mining field. Finally, this procedure enables precise validation process of a model, and it helps modification by offering feedback at the validation phase.

• Journal of Mechanical Science and Technology
• /
• v.17 no.7
• /
• pp.1083-1094
• /
• 2003

This study represents an assessment of the phase-Doppler technique to the measurements of dense droplet laden jet. High-pressure injection fuel sprays have been investigated to evaluate the use of the Phase-Doppler anemometry (PDA) technique. The critical issue is the stability of the phase-Doppler anemometry technique for dense droplet laden jet such as Diesel fuel spray in order to insure the results from the drop size and velocity measurements are repeatable, consistent, and physically realistic because the validation rate of experimental data is very low due to the thick optical density. The effect of shift frequency is minor, however, the photomultiplier tube (PMT) voltage setting is very sensitive to the data acquisition and noise in dense droplet laden jet. The optimum PMT voltage and shift frequency should be chosen so that the data such as volume flux and drop diameter do not change rapidly.