Browse > Article

On the Balanced Blending of Formally Structured and Simplified Approaches for Utilizing Judgments of Experts in the Assessment of Uncertain Issues  

Ahn Kwang-Il (Korea Atomic Energy Research Institute)
Yang Joon-Eon (Korea Atomic Energy Research Institute)
Ha Jae-Joo (Korea Atomic Energy Research Institute)
Publication Information
Nuclear Engineering and Technology / v.35, no.4, 2003 , pp. 318-335 More about this Journal
Abstract
Expert judgment is frequently employed in the search for the solution to various engineering and decision-making problems where relevant data is not sufficient or where there is little consensus as to the correct models to apply. When expert judgments are required to solve the underlying problem, our main concern is how to formally derive their technical expertise and their personal degree of familiarity about the related questions. Formal methods for gathering judgments from experts and assessing the effects of the judgments on the results of the analysis have been developed in a variety of ways. The most important interest of such methods is to establish the robustness of an expert's knowledge upon which the elicitation of judgments is made and an effective trace of the elicitation process as possible as one can. While the resultant expert judgments can remain to a large extent substantiated with formal elicitation methods, their applicability however is often limited due to restriction of available resources (e.g., time, budget, and number of qualified experts, etc) as well as a scope of the analysis. For this reason, many engineering and decision-making problems have not always performed with a formal/structured pattern, but rather relied on a pertinent transition of the formal process to the simplified approach. The purpose of this paper is (a) to address some insights into the balanced use of formally structured and simplified approaches for the explicit use of expert judgments under resource constraints and (b) to discuss related decision-theoretic issues.
Keywords
engineering and decision-making problems; uncertainty assessment; expert judgment approach; balanced use; decision-theoretic issue;
Citations & Related Records
연도 인용수 순위
  • Reference
1 G. Cojazzi, L. Pinola, and R. Sardella, 'A Benchmark Exercise on Expert Judgment Techniques in PSA Level 2: Design Criteria and General Framework,' International Topical Meeting on Probabilistic Safety Assessment (PSA '96), September 29-October 3, Park City, Utah, USA, pp.1775-1778 (1996)
2 H.Glaeser, E.Hofer, M.Kloss, T.Skorek, 'Uncertainty and Sensitivity Analysis of a Post-Experiment Calculation in Thermal Hydraulics,' Reliability Engineering and System Safety, 45, pp.19-33 (1994)   DOI   ScienceOn
3 S. Chhibber, G. Apostolakis, and D. Okrent, 'On the Use of Expert, Judgments to Estimate the Pressure Increment in the Sequoyah Containment at Vessel Breach,' Nuclear Technology, 105, pp.87-103 (1994)
4 A. Kreuser, J. Peschke, and C. Verstegen, 'Practical Application of Expert Judgment in Assessing Operation Experience for Generating Reliability Data,' CD-Rom Proceeding of PSAM 6, San Juan, Puero Rico, USA, June 24-28 (2002)
5 R.M. Cooke, 'A Theory of Weights for Combining Expert Opinion', Report 87-25, Faculty of Mathematics and Informatics/ Computer Science, Delft University of Techology, Delft, Netherlands (1987)
6 R.T. Clemen and R.L. Winkler, 'Limits fort the Precision and Value of Information from Dependent Sources,' Operational Research, 33, p.427-442 (1985)
7 A. Mosleh and G. Apostolakis, 'The Assessment of Probability Distributions from Expert Opinions with an Application to Seismic Fragility Curves, Risk Analysis, 6(4), pp.447-461 (1986)   DOI   ScienceOn
8 D.A. Seaver, D. von Winterfelt, and W. Edwards, 'Eliciting Subjective Probability Distributions on Continuous Variables,' Organizat. Behavior, Human Perform, 21, pp.379-391 (1978)   DOI
9 R.L. Winkler, 'Cornbining Probability Distributions from Dependent Information Sour ces', Management Sciences, 27, pp.(1981)   ScienceOn
10 H.A. Linstone and M. Turoff, Eds, The Delphi Method: Techniques and Applications, Reading, MA: Addison-Wesley (1975)
11 J.S. Armstrong, W.B. Denniston, and M.M. Gordon, 'The Use of the Decomposition Principle in Making Judgments,' Operational Behavior Human Perform, 14, pp.257-263 (1975)   DOI
12 A. Amendola, 'Uncertainties in Systems Reliability Modeling: Insights Gained through European Benchmark Exercises,' Nuclear Engineering and Design, 93, pp.215-225 (1986)   DOI   ScienceOn
13 G. Cojazzi, L. Pinola, U. Pulkkinen, 'The Context of the Phase I of the Benchmark Exercise on Expert Judgment Techniques in Level 2 PSA: The JRC FARO L-24 Experiment,' PSAM-4, September 13-18, New York, USA, pp.1609-1704 (1998)
14 US Nuclear Regulatory Commission, 'Severe Accident Risks: An Assessment for Five US Nuclear Power Plants', NUREG-1150, Washington, DC (1989)
15 U. Pulkkinen and R. Virolainen, 'A Formal Expert Judgment Methodology and Its Application to the Prediction of the Results of a Fuel Coolant Interaction Experiment', PSAM-4, September 13-18, New York, USA, pp.1717-1722 (1998)
16 R.L. Keeny and D. von Winterfelt, 'Eliciting Probabilities from Experts in Complex Technical Problems,' IEEE Trans. on Engineering Management, 38 (3) (August 1991)   DOI   ScienceOn
17 S.C. Hora and R.L. Iman, 'Expert Opinion in Risk Analysis: The NUREG-1150 Methodology,' Nuclear Science and Engineering, 102, pp.323-331 (1989)
18 E.J. Bonano and G. Apostolakis, 'Theoretical Foundations and Practical Issues for Using Expert Judgments in Uncertainty Analysis of High-Level Radioactive Waste Disposal,' Radioactive Waste Management on Nuclear Fuel Cycle, 16 (2), pp.137-159 (1991)
19 M.A. Meyer and J.M. Booker, Eliciting and Analyzing Expert Judgment, NUREG/CR5424, U.S. Nuclear Regulatory Commission, Washington, DC (January 1990)
20 M.W. Merkhofer, 'Quantifying Judgmental Uncertainty: Methodology, Experiences, and Insiqhts', IEEE Trans. on System, Man, and Cybernetics, SMC-17 (5), (September/October 1987)
21 G. Volta and H. Otway, 'The Logic of Probabilistic Risk Assessment versus Decision Levels', Nuclear Engineering and Design, 93, pp.329-334 (1986)   DOI   ScienceOn
22 S. Kaplan, 'Expert Information vs. Expert Opinions - Another Approach to the Problem of Eliciting/Combining/Using Expert Opinion in PRA,' Reliability Engineering and System Safety, 35, pp.61-72 (1992)   DOI   ScienceOn
23 G.W. Parry, 'A Discussion on the Use of Judgment in Representing Uncertainty in PRAs', Nuclear Engineering and Design, 93, pp.135-144 (1986)   DOI   ScienceOn
24 E. Hofer, 'On Surveys of Expert Opinion,' Nuclear Engineering and Design, 93, pp.153-160 (1986)   DOI   ScienceOn
25 A. Mosleh, 'Hidden Sources of Uncertainty: Judgment in the Collection and Analysis of Data', Nuclear Engineering and Design, 93, pp.187-198 (1986)   DOI   ScienceOn
26 A.Tversky and D.Kahneman, 'Judgment under Uncertainty: Heuristics and Biases,' Science, 185, pp.1124-1131 (1974)   DOI   ScienceOn