• Journal of Information Technology Services
• /
• v.12 no.3
• /
• pp.273-287
• /
• 2013

The objective of this study is to empirically develop performance evaluation measures for LIS (logistics information systems) based upon BSC (balanced scorecard) model and determine the relative importance among four perspectives of BSC using AHP (analytic hierarchy process) methodology. Above all, forty nine probable measures for evaluating LIS performance were identified through reviewing the previous research related with SCM (supply chain management), logistics management, and information systems evaluation. And then, these probable measures were examined by means of coincidence analyses using three mutually exclusive criteria(validity of content, ease of measurement and reliability of measurement). Data for conducting the coincidence analyses were collected from LIS users and LIS development personnel. As the results of the coincidence analyses, it was found that 31 ones among 49 probable performance evaluation measures theoretically derived could be qualified for LIS performance evaluation. And AHP analysis showed that the weight of each perspective was respectively as follows : 46.8% for financial perspective, 31.3% for customer perspective, 14.8% for internal business process perspective, and 7.1% for learning and growth perspective. The academic contribution of this study is that performance evaluation measures for LIS could be systematically and empirically developed on the basis of BSC. Also, the results of this study are expected to be used as a practice guideline of evaluating and improving LIS.

• Journal of Technology Innovation
• /
• v.14 no.3
• /
• pp.117-154
• /
• 2006

This research examines performance evaluatees' opinions regarding the current institutional performance evaluation systems of Government S&T Research Institutes (GRIs). Under the current evaluation systems, twenty GRIs are grouped into three Research Groups and each Group has its own evaluation systems. One problem of the current institutional evaluation systems is that the systems cannot reflect individual GRIs' characteristics. The following methods are used. First, based on four perspectives of Kaplan & Norton(1992)'s Balanced Scorecard(BSC) model, six perspectives appropriate to GRUs' characteristics are derived. Second, experts classify current performance evaluation measures into the six perspectives. This enables different evaluation systems of three GRI Research Groups to be compared under the same evaluation measures. Third, GRIs' evaluatees are asked to allocate appropriate weights on the performance measures. Evaluatees' weights of a GRI are compared with average weights of the related Group. Finally in every BSC's perspective, GRIs that have extraordinarily over-scored or under-scored weights are analyzed in terms of GRIs' missions, customers, capability of human resources, etc. In the Basic Research Group, the Korea Basic Science Institute is deviated in the financial perspective and the strategic direction perspectives. In the Public Research Group, Korea Institute of Construction Technology is significantly different from other GRIs in three perspectives. Five out of eight GRIs in the Industrial Research Group, GRIs are significantly different each other in several perspectives. It could be concluded that the current institutional evaluation systems are least appropriate in measuring performance of the GRIs of the Industrial Research Group.

• Journal of Information Technology Applications and Management
• /
• v.16 no.2
• /
• pp.99-120
• /
• 2009

The relationship between the performance measures and the acceptance of evaluation has been of importance not only in the academic field but in the business area. However, there had not been many studies on the factors which affect the goal core, that is, specificity and difficulty. The previous literature on the evaluation of measures has limitations in that the common and unique measures without suggesting the concept of shared measures are used. Thus, the purpose of this study is to examine the effects of the environmental characteristics of the goal setting on the result and acceptance of the evaluation, which is mediated by the characteristics of the goal core. This study further tests if there are any moderating effects from using the shared measures of performance.

• Journal of Korea Technology Innovation Society
• /
• v.8 no.3
• /
• pp.1087-1116
• /
• 2005

This research examines performance evaluatees' opinions regarding the institutional performance evaluation systems of Government S&T Research Institutes (GRI). Research methodology is as follows: first, four perspectives of Kaplan & Norton (1992) Balanced Scorecard Model are revised into six perspectives suitable to GRI's characteristics. Second, experts classify current performance evaluation measures into the six perspectives. This enables different evaluation systems of three GRI evaluation groups to be compared under the same evaluation measures. Third, GRI's evaluatees are asked to allocate ideal weights on the performance measures. The evaluatees' weights are compared with the weights of current performance measures, and the characteristics of evaluatees' opinions about current performance evaluation systems are analyzed. Results are as follows; first, six perspectives for Korean GRIs are financial, long-term outcome, short-term outcome, strategic direction, project management, human resources perspectives. second, GRI evaluation systems put the most weights on the long- and short-term outcome perspectives and the least weights on the financial perspective. This result complies with theoretical model: in performance evaluation of GRIs, the customer perspective is the most important one while the financial perspective is the least important one. Third, evaluatees think that the long-term outcome perspective of the Basic-technology GRI group and the short-term outcome perspective of the Applied-technology GRI Group needs more weight. Fourth, it is found that the current systems have more weights on the project management and strategic direction perspectives than evaluatees think. The possible explanation of this result would be that since the measures of these perspectives are relatively easier to set up than those of other perspectives, the current systems contains larger number of measures and, accordingly more weights.

• Health Policy and Management
• /
• v.9 no.2
• /
• pp.21-39
• /
• 1999

An increasing number of hospitals are seeking for new or mixed compensation strategies to improve the productivity of their medical staff in the struggle to provide high quality medical services at low costs amid the economic hardship. To motivate physicians toward the right direction, it is necessary to effectively evaluate their performance that provides a basis for compensation. However, productivity has been historically difficult to measure, particularly for physicians in academic medical centers who are expected to engage in research, education, and patient care simultaneously. The objectives of this study were to define performance measures of physicians and clinical departments in academic medical centers. to examine correlations between the measures. and to investigate factors affecting the measures. The performance data of 212 faculty physicians in 17 clinical departments in two university teaching hospitals affiliated to one medical school during the fiscal year 1994 was used for analyses. Patient care revenue, net profit. and adjusted number of patients were defined to measure the performance in patient care. and number of articles published in academic journals and research grant were defined for research activities. Both individual physicians' performance measures and per physician measures of clinical departments were analyzed. All measures defined to evaluate individual physicians' performance were positively related to each other. Clinical department and rank of faculty position were statistically significant predictors of revenue. and hospital. clinical department. and rank were significant predictors of net profit. journal publication. and research grant. Patient care measures defined to evaluate clinical departments were related to each other. so were research measures. and no significant correlations were found between patient care measures and research measures. Also found were large differences in department. ranks when clinical departments were evaluated by absolute per physician performance measures and evaluated by annual rate of changes in performance measures. These findings suggest that departmental performance measures opposed to individual performance measures are relatively free from problems of factors affecting the performance measures that are not in control of clinical departments or individual physicians. Results from the correlation analysis of departmental performance measures indicates that measures of research performance should be included in the evaluation to promote research activities in academic medical centers.

• Proceedings of the Korean Operations and Management Science Society Conference
• /
• 2004.05a
• /
• pp.163-166
• /
• 2004

This paper proposes decomposition-based approach for developing the evaluation factors of performance measures and designing performance measures in manufacturing system. In this paper, the evaluation factors are designed by design decomposition approach and the design process of performance measures is based on the manufacturing system design decomposition.

• Journal of Korea Technology Innovation Society
• /
• v.1 no.1
• /
• pp.117-124
• /
• 1998

Corporations are pursuing maximum returns from their R&D investment. They are also interested in sound measures to quantify returns. In fact, they use various measures and criteria for measuring returns from the R&D investment. But the fundamental problem is that there is no generic and widely acceptable measures and criteria. To make things more complicated, measures are very powerful and influential to the people in the corporations. Herbert Simon already indicated that people do many things but people usually do their best for the only tasks which are measured. Many researchers, like Chester(1995), are interested in R&D productivity measures and risks because what the company measures really influence R&D people and output. This article present design concepts of the R&D project selection and evaluation system in POSCO(Pohang Iron & Steel Company). This is an output extract from the 6-month joint activities with POSRI(POSCO Research Institute) researchers and POSCO R&D personnel. Process changes, new organizations and new selection and evaluation criteria are developed to improve R&D performance and to enhance technology management of the POSCO. This article covers new selection and evaluation criteria only. We would like to share our experience about how we redesign the selection and evaluation of R&D projects. We also bring insights how we seamlessly integrate 4 different project selection and evaluation steps as a whole. We hope that this case will give you a clue to improve your R&D management.

• Journal of the Ergonomics Society of Korea
• /
• v.25 no.3
• /
• pp.85-96
• /
• 2006

Main control room(MCR) man-machine interface(MMI) design of advanced nuclear power plants(NPPs) such as APR(advanced power reactor)-1400 can be validated through performance-based tests to determine whether it acceptably supports safe operation of the plant. In this work, plant performance, personnel task, situation awareness, workload, teamwork, and anthropometric/physiological factor are considered as factors for the human performance evaluation. For development of measures in each of the factors, techniques generally used in various industries and empirically proven to be useful are adopted as main measures and some helpful techniques are developed in order to complement the main measures. Also the development of the measures is addressed based on the theoretical background. Finally we discuss the way in which the measures can be effectively integrated and then HUPESS(HUman Performance Evaluation Support System) which is in development based on the integrated way is briefly introduced.

• Journal of the Korean BIBLIA Society for library and Information Science
• /
• v.14 no.1
• /
• pp.79-90
• /
• 2003

• Proceedings of the Technology Innovation Conference
• /
• 1997.07a
• /
• pp.247-265
• /
• 1997

The objectives of not-for-profit organizations including government supported research institutes are, by definition, to achieve socially desired nonfinancial goals. Current reporting focuses on providing information to meet the needs of users. This information may be provided in the annual research report, financial report, and other reports. An objective of current reporting is to provide users with information that will assist them in evaluating the performance(efficiency and effectiveness) of reporting entity. The evaluation of R&D project's performance requires information not only about the acquition and use of resources, but also about the outputs and outcomes of R&D activities, This study aims to recommend that R&D Service Efforts and Accomplish- mints(SEA) reporting is useful for performance evaluation in national R&D programmes. To achieve this aims, I attempt to develop the R&D SEA measures with reference to the Concepts Statement No. 2 of the Governmental Accounting Standards Board. R&D SEA measures consist of five categories : (1) input measures, (2) outcome measures, (3) output measures, (4) re1a1e efforts to accomplishments, (5) explanatory information.