• International Journal of Computer Science & Network Security
• /
• 제21권12spc호
• /
• pp.549-555
• /
• 2021

Machine Learning(ML) splits data into 3 parts, which are usually 60% for training, 20% for validation, and 20% for testing. It just splits quantitatively instead of selecting each set of data by a criterion, which is very important concept for the adequacy of test data. ML measures a model's accuracy by applying a set of validation data, and revises the model until the validation accuracy reaches on a certain level. After the validation process, the complete model is tested with the set of test data, which are not seen by the model yet. If the set of test data covers the model's attributes well, the test accuracy will be close to the validation accuracy of the model. To make sure that ML's set of test data works adequately, we design an experiment and see if the test accuracy of model is always close to its validation adequacy as expected. The experiment builds 100 different SVM models for each of six data sets published in UCI ML repository. From the test accuracy and its validation accuracy of 600 cases, we find some unexpected cases, where the test accuracy is very different from its validation accuracy. Consequently, it is not always true that ML's set of test data is adequate to assure a model's quality.

• 한국소음진동공학회:학술대회논문집
• /
• 한국소음진동공학회 2014년도 추계학술대회 논문집
• /
• pp.347-349
• /
• 2014

Model verification and validation (V&V) is a current research topic to build computational models with high predictive capability by addressing the general concepts, processes and statistical techniques. The hypothesis test for validity check is one of the model validation techniques and gives a guideline to evaluate the validity of a computational model when limited experimental data only exist due to restricted test resources (e.g., time and budget). The hypothesis test for validity check mainly employ Type I error, the risk of rejecting the valid computational model, for the validity evaluation since quantification of Type II error is not feasible for model validation. However, Type II error, the risk of accepting invalid computational model, should be importantly considered for an engineered products having high risk on predicted results. This paper proposes a technique named as the response-adaptive experimental design to reduce Type II error by adaptively designing experimental conditions for the validation experiment. A tire tread block problem and a numerical example are employed to show the effectiveness of the response-adaptive experimental design for the validity evaluation.

• 대한기계학회논문집A
• /
• 제31권1호
• /
• pp.55-61
• /
• 2007

Recently simulation model becomes an essential tool for analysis and design of a system but it is often expensive and time consuming as it becomes complicate to achieve reliable results. Therefore, high-fidelity simulation model needs to be replaced by an approximate model, the so-called metamodel. Metamodeling techniques include 3 components of sampling, metamodel and validation. Cross-validation approach has been proposed to provide sequnatially new sample point based on cross-validation error but it is very expensive because cross-validation must be evaluated at each stage. To enhance the cross-validation of metamodel, sequential sampling method using candidate points and representative cross-validation is proposed in this paper. The candidate and representative cross-validation approach of sequential sampling is illustrated for two-dimensional domain. To verify the performance of the suggested sampling technique, we compare the accuracy of the metamodels for various mathematical functions with that obtained by conventional sequential sampling strategies such as maximum distance, mean squared error, and maximum entropy sequential samplings. Through this research we team that the proposed approach is computationally inexpensive and provides good prediction performance.

• 응용통계연구
• /
• 제25권1호
• /
• pp.139-152
• /
• 2012

집단 약동학 모형 추정의 결과는 환자에게 투약학 약물의 용량결정에 직접적 영향을 미치므로 추정 모형에 대한 타당도와 적합도의 검증이 중요하다. 본 논문에서는 다양한 집단 약동학 모형 적합도 검증을 위한 방법들을 비교, 분석하고 실제 임상자료를 이용하여 최적의 집단 약동학 모형을 찾고 이에 대하여 다양한 타당도, 적합도 검정을 실시하여 모형을 진단해 본다.

• Steel and Composite Structures
• /
• 제17권6호
• /
• pp.835-850
• /
• 2014

The fire resistance of composite steel and concrete structures may be determined by using the simplified methods provided in EN 1994-1-2. For the particular situations not covered by the standard, an advanced calculation model might be applied, using special purpose programs for the analysis of structures in fire. The validation of these programs has always been an important issue for software developers, but also for designers and authorities. Clause 4.4.4 from EN 1994-1-2 refers to the validation of the advanced calculation models and states that these models must be validated through relevant test results. The paper presents the calculation of fire resistance of the composite columns in a high-rise building built in Romania, and focusses on the validation of the calculation model (computer program SAFIR), for this particular case. This validation, asked by the Romanian authorities, considers the available experimental results of a fire test, performed on a similar composite steel-concrete column.

• Geomechanics and Engineering
• /
• 제28권5호
• /
• pp.437-454
• /
• 2022

The hydro-mechanical (HM) two-way sequential coupling in the TOUGH2-FLAC3D linking algorithm is validated completely and successfully in both M to H and H to M directions, which are initiated by mechanical surface loading for geomechanical validation and hydrological groundwater pumping for hydrogeological validation, respectively. For such complete and successful validation, a TOUGH2-FLAC3D linked numerical model is developed first by adopting the TOUGH2-FLAC3D linking algorithm, which uses the two-way (fixed-stress split) sequential coupling scheme and the implicit backward time stepping method. Two geomechanical and two hydrogeological validation problems are then simulated using the linked numerical model together with basic validation strategies and prerequisites. The second geomechanical and second hydrogeological validation problems are also associated with the Mandel effect and the Noordbergum and Rhade effects, respectively, which are three phenomenally well-known but numerically challenging HM effects. Finally, sequentially coupled numerical solutions are compared with either analytical solutions (verification) or fully coupled numerical solutions (benchmarking). In all the four validation problems, they show almost perfect to extremely or very good agreement. In addition, the second geomechanical validation problem clearly displays the Mandel effect and suggests a proper or minimum geometrical ratio of the height to the width for the rectangular domain to maximize agreement between the numerical and analytical solutions. In the meantime, the second hydrogeological validation problem clearly displays the Noordbergum and Rhade effects and implies that the HM two-way sequential coupling scheme used in the linked numerical model is as rigorous as the HM two-way full coupling scheme used in a fully coupled numerical model.

• Korean Chemical Engineering Research
• /
• 제59권4호
• /
• pp.532-541
• /
• 2021

최근 인공지능에 대한 관심이 높아짐에 따라 화학공정분야에서도 인공지능을 활용한 연구가 많아지고 있다. 그러나 인공지능 기반 모델이 충분히 일반화되지 않아 학습에 이용되지 않은 새로운 데이터에 대한 예측률이 떨어지는 과적합 현상이 빈번하게 일어나고 있으며, 교차검증은 과적합을 해결하는 방법 중 하나이다. 본 연구에서는 2,3-BDO 분리 공정 온도 예측 모델의 초매개변수 중에서 배치 개수와 반복횟수를 조정하기 위해 시계열 교차검증을 적용하고 일반적으로 사용되는 K 겹 교차검증과 비교하였다. 결과적으로 K 겹 교차검증을 사용했을 때 보다 시계열 교차검증 방식을 사용했을 때 MAPE는 0.61% 증가한 반면 RMSE는 9.06% 감소하였고 학습 시간은 198.29초 적게 소요되었다.

• Coupled systems mechanics
• /
• 제4권1호
• /
• pp.37-65
• /
• 2015

Modeling and simulation of mechanical response of infrastructure object, solids and structures, relies on the use of computational models to foretell the state of a physical system under conditions for which such computational model has not been validated. Verification and Validation (V&V) procedures are the primary means of assessing accuracy, building confidence and credibility in modeling and computational simulations of behavior of those infrastructure objects. Validation is the process of determining a degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model. It is mainly a physics issue and provides evidence that the correct model is solved (Oberkampf et al. 2002). Our primary interest is in modeling and simulating behavior of porous particulate media that is fully saturated with pore fluid, including cyclic mobility and liquefaction. Fully saturated soils undergoing dynamic shaking fall in this category. Verification modeling and simulation of fully saturated porous soils is addressed in more detail by (Tasiopoulou et al. 2014), and in this paper we address validation. A set of centrifuge experiments is used for this purpose. Discussion is provided assessing the effects of scaling laws on centrifuge experiments and their influence on the validation. Available validation test are reviewed in view of first and second order phenomena and their importance to validation. For example, dynamics behavior of the system, following the dynamic time, and dissipation of the pore fluid pressures, following diffusion time, are not happening in the same time scale and those discrepancies are discussed. Laboratory tests, performed on soil that is used in centrifuge experiments, were used to calibrate material models that are then used in a validation process. Number of physical and numerical examples are used for validation and to illustrate presented discussion. In particular, it is shown that for the most part, numerical prediction of behavior, using laboratory test data to calibrate soil material model, prior to centrifuge experiments, can be validated using scaled tests. There are, of course, discrepancies, sources of which are analyzed and discussed.

• 한국컴퓨터정보학회논문지
• /
• 제27권11호
• /
• pp.13-18
• /
• 2022

본 논문에서는 학습에 참여하는 각 디바이스의 모델들로부터 성능검증에 따라 가중치를 두어 글로벌 모델을 업데이트하는 VW-FedAVG(Validation based Weighted FedAVG)를 두 가지 방식으로 제안 한다. 첫 번째 방식은 서버 검증(Server side Validation) 구조로 글로벌 모델을 업데이트 하기 전에 각 로컬 클라이언트 모델을 하나의 전체 검증 데이터셋을 통해 검증하도록 설계 했다. 두 번째는 클라이언트 검증(Client side Validation) 구조로 검증 데이터셋을 각 클라이언트에 고르게 분배하여 검증을 한 후 글로벌 모델을 업데이트 하는 방식으로 설계 했다. 전체 실험에 적용한 데이터셋은 MNIST, CIFAR-10으로 이미지 분류에 대해 IID, Non-IID 분포에서 기존 연구 대비 더 높은 정확도를 얻을 수 있었다.

• 대한산업공학회지
• /
• 제6권1호
• /
• pp.27-38
• /
• 1980

This paper presents the validation of a stochastic model of muscle fatigue during static muscle contractions. Forty four laboratory experiments, covering eleven test conditions for two trained subjects, were run in order to estimate fatigue and recovery rates, based on EMG observations. The validation of the model was made by comparing the model predictions to the experimental fatigue time. The validation study supports that the stochastic model of muscle fatigue accurately represents the underlying fatigue process. The study also provides support that the fatigue model can be used as a monitor of individual muscle capabilities.