• Journal of Technology Innovation
• /
• v.14 no.3
• /
• pp.117-154
• /
• 2006

This research examines performance evaluatees' opinions regarding the current institutional performance evaluation systems of Government S&T Research Institutes (GRIs). Under the current evaluation systems, twenty GRIs are grouped into three Research Groups and each Group has its own evaluation systems. One problem of the current institutional evaluation systems is that the systems cannot reflect individual GRIs' characteristics. The following methods are used. First, based on four perspectives of Kaplan & Norton(1992)'s Balanced Scorecard(BSC) model, six perspectives appropriate to GRUs' characteristics are derived. Second, experts classify current performance evaluation measures into the six perspectives. This enables different evaluation systems of three GRI Research Groups to be compared under the same evaluation measures. Third, GRIs' evaluatees are asked to allocate appropriate weights on the performance measures. Evaluatees' weights of a GRI are compared with average weights of the related Group. Finally in every BSC's perspective, GRIs that have extraordinarily over-scored or under-scored weights are analyzed in terms of GRIs' missions, customers, capability of human resources, etc. In the Basic Research Group, the Korea Basic Science Institute is deviated in the financial perspective and the strategic direction perspectives. In the Public Research Group, Korea Institute of Construction Technology is significantly different from other GRIs in three perspectives. Five out of eight GRIs in the Industrial Research Group, GRIs are significantly different each other in several perspectives. It could be concluded that the current institutional evaluation systems are least appropriate in measuring performance of the GRIs of the Industrial Research Group.

• Korean Journal of Social Welfare
• /
• v.45
• /
• pp.126-155
• /
• 2001

It has been 5 years since evaluation systems were introduced to the social welfare field in Korea. More than half of social welfare agencies (617 out of 1,267) were evaluated in the past two years. Most areas of the agencies were the target of an extensive evaluation in 2000. As "the Evaluation Era" has come, problems were raised especially regarding the Accreditation system of the Ministry of Health and Social Welfare. More specifically, the problems were related to operation and management of the evaluation systems including the issue of evaluators' objectivity and fairness. In addition, evaluation costs have been too high to the government. The evaluation systems of OECD nations including the United States were quite different. These nations have used the Performance Measurement in order to secure the objectivity and the fairness. Although the quantity of information was limited compared to that of the Accreditation system, these nations have taken the lead in the governmental evaluation. In this context, this study compared social welfare evaluation systems between Korea and the United States. It presented the current status and problems of the Accreditation system of the Ministry of Health and Social Welfare, discussed the American Performance Measurement and its limitations, and compared the strengths and weaknesses of the two evaluation systems. In addition, this study suggested the strategies of Korean evaluation systems in the long term as well as in the short term.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.12 no.3
• /
• pp.61-76
• /
• 2002

To ensure PKI systems' reliability, the security for PKI systems evaluation is required. But, unfortunately, the systematic security evaluation and certification of PKI systems is insufficient. In Korea, Firewall and intrusion detection system's security evaluation and certification has been enforced, but research of PKI systems’ evaluation is insufficient. This paper provides a PKI system evaluation criteria. This paper specifies a 7 level of the functional and assurance security requirements for a PKI system. And this PKI system evaluation criteria provides a compatibility with CC(Common Criteria) and KISES(Korea Information Security Evaluation Systems).

• Journal of Information Science Theory and Practice
• /
• v.3 no.4
• /
• pp.19-34
• /
• 2015

Evaluation of retrieval systems is mostly limited to laboratory settings and rarely considers changes of performance over time. This article presents an evaluation of retrieval systems for internal Web site search systems between the years 2006 and 2011. A holistic evaluation methodology for real Web sites was developed which includes tests for functionality, search quality, and user interaction. Among other sites, one set of 20 Web site search systems was evaluated three times in different years and no substantial improvement could be shown. It is surprising that the communication between site and user still leads to very poor results in many cases. Overall, the quality of these search systems could be improved, and several areas for improvement are apparent from our evaluation. For a comparison, Google’s site search function was also tested with the same tasks.

• Journal of the Korea Institute of Military Science and Technology
• /
• v.12 no.4
• /
• pp.424-436
• /
• 2009

In the modern warfield, as the defense systems has been digitalized, intelligent, and working with network, the importance of interoperability among the systems has been also increasing. KMND(Korean Ministry of National Defense) has developed and used the SITES(Systems Interoperability Test & Evaluation System) based on Korean LISI(Levels of Information Systems Interoperability). SITES had been developed in order to evaluate the interoperability among information systems from the viewpoint of technical attributes. In this paper, we propose the extended interoperability evaluation model which can be applied not only the information systems, but also the weapon systems. Moreover, the extended interoperability evaluation model also includes the consideration of non-technical attributes.

• International Journal of Safety
• /
• v.11 no.1
• /
• pp.26-32
• /
• 2012

This paper introduces a feasibility evaluation method for prognosis systems based on an empirical model in nuclear power plants. By exploiting the dynamical signature characterized by abnormal phenomena, the prognosis technique can be applied to detect the plant abnormal states prior to an unexpected plant trip. Early $operator^{\circ}{\emptyset}s$ awareness can extend available time for operation action; therefore, unexpected plant trip and time-consuming maintenance can be reduced. For the practical application in nuclear power plant, it is important not only to enhance the advantages of prognosis systems, but also to quantify the negative impact in prognosis, e.g., uncertainty. In order to apply these prognosis systems to real nuclear power plants, it is necessary to conduct a feasibility evaluation; the evaluation consists of 4 steps (: the development of an evaluation method, the development of selection criteria for the abnormal state, acquisition and signal processing, and an evaluation experiment). In this paper, we introduce the feasibility evaluation method and propose further study points for applying prognosis systems from KHNP's experiences in testing some prognosis technologies available in the market.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2014.07a
• /
• pp.97-98
• /
• 2014

Lots of investment projects of new development and redevelopment for information systems have been not taken care of in the field of administration and evaluation, for these information systems projects have unique characteristics such as technology sensitiveness, network effectiveness, embeddedness, and externality. In fact, quantitative and qualitative evaluation of investments in information systems projects are not sufficient. It is critically important to generally evaluate benefits of development or operation cost, urgency, external effects, and so on. In addition, the efficient monitoring and effective analysis of information systems are surely needed for beneficient results of investment in information systems. We propose an economics evaluation model for information systems projects.

• Journal of Korea Multimedia Society
• /
• v.13 no.4
• /
• pp.487-495
• /
• 2010

It has been required to develop an accurate and reliable evaluation method for the performance of biometric systems as their use is getting popular. Among a number of biometric systems, face recognition is one of the most widely used techniques and this leads to develop a stable evaluation method for face recognition systems in order to standardize the performance of face recognition systems. However, it is considered as a difficult task to evaluation such systems due to a large number of factors that affect their performance. Thus, it may be infeasible to take into account all the environmental factors that are related to the performance of face recognition systems and this naturally suggests an evaluation method for the overall performance based on scenarios. In this paper, we have analyzed environmental factors that are related to the performance of general face recognition systems and proposed their evaluation method taking into account those factors. We have proposed an evaluation method based on scenario that considers the combination of individual environment factors instead of evaluating the performance of face recognition systems regarding each factor. Indeed, we have presented examples on the evaluation of face recognition systems based on scenario that takes into account overall environmental factors.

• The Transactions of the Korean Institute of Electrical Engineers A
• /
• v.49 no.4
• /
• pp.152-160
• /
• 2000

This paper presents an efficient evaluation method on the role of new energy storage systems, especially the secondary Battery Energy Storage (BES) systems, in the case where they are interconnected with the power distribution systems. It is important to perform the economic evaluation for the new energy storage systems in a synthetical and quantitative manner, because they are very costly in the early stage of their development and commercialization. In this paper, the multiple functions of BES systems, which are operated at distribution systems, such as load levelling, effective utilization of power distribution systems and uninterruptible power supply at the emergency conditions are classified and analyzed. And then the quantitative evaluation methods of the multiple functions for BES systems are proposed using the mathematical modelling.

• Proceedings of the Korea Concrete Institute Conference
• /
• 2000.10b
• /
• pp.1291-1296
• /
• 2000

A basis for the direct use of data from nondestructive evaluation methods in bridge management systems is presented. Bridge management systems use integer-valued condition ratings to recognize conditions of bridge elements, to model progression of deterioration, and to determine repair needs. Data from nondestructive evaluation methods can inform management systems on the extent of damage, on the initiation of deterioration processes, and on the exposure of bridge elements to aggressive agents. In addition, data obtained through nondestructive evaluation methods allow the formation of models of specific deterioration process. The use of these data in bridge management systems requires redefinition of condition ratings together with the creation of procedures for automated interpretation of data. By these action, nondestructive evaluation methods are directly used to assign condition ratings, and condition ratings are made into terse form of NDE data that are compatible with present day bridge management systems. This paper reports work in progress to strategic use of nondestructive evaluation methods in bridge management system.