• International Journal of Computer Science & Network Security
• /
• v.21 no.12spc
• /
• pp.549-555
• /
• 2021

Machine Learning(ML) splits data into 3 parts, which are usually 60% for training, 20% for validation, and 20% for testing. It just splits quantitatively instead of selecting each set of data by a criterion, which is very important concept for the adequacy of test data. ML measures a model's accuracy by applying a set of validation data, and revises the model until the validation accuracy reaches on a certain level. After the validation process, the complete model is tested with the set of test data, which are not seen by the model yet. If the set of test data covers the model's attributes well, the test accuracy will be close to the validation accuracy of the model. To make sure that ML's set of test data works adequately, we design an experiment and see if the test accuracy of model is always close to its validation adequacy as expected. The experiment builds 100 different SVM models for each of six data sets published in UCI ML repository. From the test accuracy and its validation accuracy of 600 cases, we find some unexpected cases, where the test accuracy is very different from its validation accuracy. Consequently, it is not always true that ML's set of test data is adequate to assure a model's quality.

• Journal of the Korea Society for Simulation
• /
• v.1 no.1
• /
• pp.77-86
• /
• 1992

In this paper, we study validation test methods of DEVSA(Descrete Event system Specification) models using SPN(Stochastic Petri Net) models. We discuss conventional validation test methods, by which DEVS models can be transformed to SPN models, by reviewing the features of DEVS model. Based on the model transformation method, we define a new homogeneous function for validation test and suggest a new validation test method of DEVS models using the property of SPN models and the new homogeneous function.

• 시스템엔지니어링워크숍
• /
• s.1
• /
• pp.27-31
• /
• 2003

This study is in a part of requirements validation plan for korean rubber-tired AGT system on test track. The AGT system is consisted subsystems as vehicle, signalling, communication, power distribution and infrastructure for rubber tire running on track. The subsystems will be installed and integrated on test track till next year for test and evaluation. This paper shows overview for test and evaluation in terms of system requirements and its validation classification, test track configuration, measuring system requirements and its configuration. The whole process of system integration and its validation will be controlled by means of KMS including documentation.

• Proceedings of the Korean Society for Noise and Vibration Engineering Conference
• /
• 2014.10a
• /
• pp.347-349
• /
• 2014

Model verification and validation (V&V) is a current research topic to build computational models with high predictive capability by addressing the general concepts, processes and statistical techniques. The hypothesis test for validity check is one of the model validation techniques and gives a guideline to evaluate the validity of a computational model when limited experimental data only exist due to restricted test resources (e.g., time and budget). The hypothesis test for validity check mainly employ Type I error, the risk of rejecting the valid computational model, for the validity evaluation since quantification of Type II error is not feasible for model validation. However, Type II error, the risk of accepting invalid computational model, should be importantly considered for an engineered products having high risk on predicted results. This paper proposes a technique named as the response-adaptive experimental design to reduce Type II error by adaptively designing experimental conditions for the validation experiment. A tire tread block problem and a numerical example are employed to show the effectiveness of the response-adaptive experimental design for the validity evaluation.

• Steel and Composite Structures
• /
• v.17 no.6
• /
• pp.835-850
• /
• 2014

The fire resistance of composite steel and concrete structures may be determined by using the simplified methods provided in EN 1994-1-2. For the particular situations not covered by the standard, an advanced calculation model might be applied, using special purpose programs for the analysis of structures in fire. The validation of these programs has always been an important issue for software developers, but also for designers and authorities. Clause 4.4.4 from EN 1994-1-2 refers to the validation of the advanced calculation models and states that these models must be validated through relevant test results. The paper presents the calculation of fire resistance of the composite columns in a high-rise building built in Romania, and focusses on the validation of the calculation model (computer program SAFIR), for this particular case. This validation, asked by the Romanian authorities, considers the available experimental results of a fire test, performed on a similar composite steel-concrete column.

• Transactions of the Korean Society of Automotive Engineers
• /
• v.25 no.3
• /
• pp.344-349
• /
• 2017

Suspension noise from under a passenger car is one of the important factors that impact the perceptual quality for drivers. However, it is difficult to validate this by component level testing in the early stage of development, because suspension noise caused by interaction of the related parts has been found at saleable vehicles late during development or at the manufacturing stage, when many customers have already filed for claims. This study proposed a validation testing under research by the DFSS process that enables reproduction of vehicle level noise by component level testing using a shock absorber with the related parts, such as urethane bumper and top mount. This study also developed a compromised test matrix while analyzing the noise factors through experimental design and analysis of variance to determine what factors can affect noise. Based on this study, we expect that the vehicle level and customer claim can be validated during initial development timing by a more reliable component noise validation test.

• Journal of the Korean Society for Aviation and Aeronautics
• /
• v.14 no.2
• /
• pp.9-16
• /
• 2006

FAA AC120-40B Level D flight dynamics model for T-50 Full Mission Trainer was successfully developed. Since AC120-40B Level D requires the quantitative validation tests for simulation model compared with flight test data, T-50 flight test data for each validation test item was gathered, and also automatic test environments which include AFT (Automatic Fidelity Tester) and STA (Simulation Test Analyzer) were developed. The final test results after the iterative test-tuning processes were all within the tolerances specified in AC120-40B Level D. Qualification Test Guide, QTG contains the detail test processes and results.

• Journal of the Society of Cosmetic Scientists of Korea
• /
• v.25 no.2
• /
• pp.129-138
• /
• 1999

피부 전용 제재 개발을 위해 요구되는 동물 대체 시험법 중 가장 적극적으로 연구가 행하여지곤 실제 실용화가 예상되는 것은 안점막 자극 시험으로 지금까지 여러 가지 방법이 개발되었지만 그 중 계란 유정란의 응모요막(CAM)을 이용한 방법이 현재 가장 활발히 진행되고 있다. 이 방법이 일부 국가에서 이미 안점막 자극 시험 동물 대체 시험법으로 공인되었으며 현재까지도 validation 연구를 활발히 진행하고 있다. 본 연구에서도 국내에 적합한 안점막 자극 시험 동물 대체 시험법의 공인 시험법 개발 및 validation study를 목표로 계란 유정란의 응모요막을 이용한 방법 중 HET-CAM 방법을 시행하였으며 안점막 동물 대체 시험법으로 확립하고자 하였다. 틴ET-CAM 방법의 보완을 위해 배양된 세포를 통해 자극도를 측정할 수 있는 방법인 Cytotoxicity test를 도입하여 시행하였으며 두 방법의 data들을 분석하여 validation study를 수행하였다. 국내 유수의 6개 장업사가 본 연구에 참가하여 20가지의 화장품 전용제재를 대상으로 1차, 2차 validation study 를 진행하였다. HET-CAM test, Draize eye irritation test, Cytotoxicity test 측정 결과 HET-CAM 의 “Q” 수치는 대부분 강자극 수치인 2 이상이었고 10% sodium hydroxide가 가장 높은 수치를 보였으며 Tween 20(sorbitanpolyoxyethylene monolaurate) 100%가 가장 낮은 수치를 보였다. In vi패의 경우 10% sodium hydroxide가 가장 높은 수치를 보였으며 30군 propylene glycol 이 가장 낮은 자극수치를 보였다. HET-CAM test 와 Draize eye irritation test, Cytotoxicity test 간의 상관성 분석은 linear correlation coefficient 와 rank correlation coefficient를 구하여 비교하였으며 6개 장업사(A-F)의 실험실에서의 HET-CAM test 결과를 취합하여 각각 두 실험실간의 상관관계(linear correlation)를 분석하였다. Linear correlation coefficient 분석 결과를 보면 전반적으로 상관관계가 0.589 - 0.954의 범위였으며, 특히 A사와 B사 사이의 경우 0.954이었으며, E사와 D사 사이의 경우 0.942로 높은 상관관계를 보였다. 그 외에도 A사와 D사 사이의 경우(0.589)와 B사와 D사 사이의 경우(0.638)를 제외하고는 대체로 높은 상관관계를 나타내었다.

• Coupled systems mechanics
• /
• v.4 no.1
• /
• pp.37-65
• /
• 2015

Modeling and simulation of mechanical response of infrastructure object, solids and structures, relies on the use of computational models to foretell the state of a physical system under conditions for which such computational model has not been validated. Verification and Validation (V&V) procedures are the primary means of assessing accuracy, building confidence and credibility in modeling and computational simulations of behavior of those infrastructure objects. Validation is the process of determining a degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model. It is mainly a physics issue and provides evidence that the correct model is solved (Oberkampf et al. 2002). Our primary interest is in modeling and simulating behavior of porous particulate media that is fully saturated with pore fluid, including cyclic mobility and liquefaction. Fully saturated soils undergoing dynamic shaking fall in this category. Verification modeling and simulation of fully saturated porous soils is addressed in more detail by (Tasiopoulou et al. 2014), and in this paper we address validation. A set of centrifuge experiments is used for this purpose. Discussion is provided assessing the effects of scaling laws on centrifuge experiments and their influence on the validation. Available validation test are reviewed in view of first and second order phenomena and their importance to validation. For example, dynamics behavior of the system, following the dynamic time, and dissipation of the pore fluid pressures, following diffusion time, are not happening in the same time scale and those discrepancies are discussed. Laboratory tests, performed on soil that is used in centrifuge experiments, were used to calibrate material models that are then used in a validation process. Number of physical and numerical examples are used for validation and to illustrate presented discussion. In particular, it is shown that for the most part, numerical prediction of behavior, using laboratory test data to calibrate soil material model, prior to centrifuge experiments, can be validated using scaled tests. There are, of course, discrepancies, sources of which are analyzed and discussed.

• Journal of Digital Contents Society
• /
• v.18 no.3
• /
• pp.525-534
• /
• 2017

The open set recognition method should be used for the cases that the classes of test data are not known completely in the training phase. So it is required to include two processes of classification and the validation test. This kind of research is very necessary for commercialization of face recognition modules, but few domestic researches results about it have been published. In this paper, we propose an open set face recognition method that includes two sequential validation phases. In the first phase, with dummy classes we perform classification based on sparse representation. Here, when the test data is classified into a dummy class, we conclude that the data is invalid. If the data is classified into one of the regular training classes, for second validation test we extract four features and apply them for the proposed decision function. In experiments, we proposed a simulation method for open set recognition and showed that the proposed validation test outperform SCI of the well-known validation method