Nuclear Engineering and Technology

Korean Nuclear Society (한국원자력학회)
권호 표지

On the Balanced Blending of Formally Structured and Simplified Approaches for Utilizing Judgments of Experts in the Assessment of Uncertain Issues

  • Published : 2003.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

Expert judgment is frequently employed in the search for the solution to various engineering and decision-making problems where relevant data is not sufficient or where there is little consensus as to the correct models to apply. When expert judgments are required to solve the underlying problem, our main concern is how to formally derive their technical expertise and their personal degree of familiarity about the related questions. Formal methods for gathering judgments from experts and assessing the effects of the judgments on the results of the analysis have been developed in a variety of ways. The most important interest of such methods is to establish the robustness of an expert's knowledge upon which the elicitation of judgments is made and an effective trace of the elicitation process as possible as one can. While the resultant expert judgments can remain to a large extent substantiated with formal elicitation methods, their applicability however is often limited due to restriction of available resources (e.g., time, budget, and number of qualified experts, etc) as well as a scope of the analysis. For this reason, many engineering and decision-making problems have not always performed with a formal/structured pattern, but rather relied on a pertinent transition of the formal process to the simplified approach. The purpose of this paper is (a) to address some insights into the balanced use of formally structured and simplified approaches for the explicit use of expert judgments under resource constraints and (b) to discuss related decision-theoretic issues.

Keywords

References

  1. A.Tversky and D.Kahneman, 'Judgment under Uncertainty: Heuristics and Biases,' Science, 185, pp.1124-1131 (1974) https://doi.org/10.1126/science.185.4157.1124
  2. G.W. Parry, 'A Discussion on the Use of Judgment in Representing Uncertainty in PRAs', Nuclear Engineering and Design, 93, pp.135-144 (1986) https://doi.org/10.1016/0029-5493(86)90212-8
  3. E. Hofer, 'On Surveys of Expert Opinion,' Nuclear Engineering and Design, 93, pp.153-160 (1986) https://doi.org/10.1016/0029-5493(86)90214-1
  4. A. Mosleh, 'Hidden Sources of Uncertainty: Judgment in the Collection and Analysis of Data', Nuclear Engineering and Design, 93, pp.187-198 (1986) https://doi.org/10.1016/0029-5493(86)90218-9
  5. G. Volta and H. Otway, 'The Logic of Probabilistic Risk Assessment versus Decision Levels', Nuclear Engineering and Design, 93, pp.329-334 (1986) https://doi.org/10.1016/0029-5493(86)90231-1
  6. S. Kaplan, 'Expert Information vs. Expert Opinions - Another Approach to the Problem of Eliciting/Combining/Using Expert Opinion in PRA,' Reliability Engineering and System Safety, 35, pp.61-72 (1992) https://doi.org/10.1016/0951-8320(92)90023-E
  7. E.J. Bonano and G. Apostolakis, 'Theoretical Foundations and Practical Issues for Using Expert Judgments in Uncertainty Analysis of High-Level Radioactive Waste Disposal,' Radioactive Waste Management on Nuclear Fuel Cycle, 16 (2), pp.137-159 (1991)
  8. M.W. Merkhofer, 'Quantifying Judgmental Uncertainty: Methodology, Experiences, and Insiqhts', IEEE Trans. on System, Man, and Cybernetics, SMC-17 (5), (September/October 1987)
  9. S.C. Hora and R.L. Iman, 'Expert Opinion in Risk Analysis: The NUREG-1150 Methodology,' Nuclear Science and Engineering, 102, pp.323-331 (1989)
  10. M.A. Meyer and J.M. Booker, Eliciting and Analyzing Expert Judgment, NUREG/CR5424, U.S. Nuclear Regulatory Commission, Washington, DC (January 1990)
  11. R.L. Keeny and D. von Winterfelt, 'Eliciting Probabilities from Experts in Complex Technical Problems,' IEEE Trans. on Engineering Management, 38 (3) (August 1991) https://doi.org/10.1109/17.83752
  12. U. Pulkkinen and R. Virolainen, 'A Formal Expert Judgment Methodology and Its Application to the Prediction of the Results of a Fuel Coolant Interaction Experiment', PSAM-4, September 13-18, New York, USA, pp.1717-1722 (1998)
  13. US Nuclear Regulatory Commission, 'Severe Accident Risks: An Assessment for Five US Nuclear Power Plants', NUREG-1150, Washington, DC (1989)
  14. G. Cojazzi, L. Pinola, U. Pulkkinen, 'The Context of the Phase I of the Benchmark Exercise on Expert Judgment Techniques in Level 2 PSA: The JRC FARO L-24 Experiment,' PSAM-4, September 13-18, New York, USA, pp.1609-1704 (1998)
  15. A. Amendola, 'Uncertainties in Systems Reliability Modeling: Insights Gained through European Benchmark Exercises,' Nuclear Engineering and Design, 93, pp.215-225 (1986) https://doi.org/10.1016/0029-5493(86)90221-9
  16. H.A. Linstone and M. Turoff, Eds, The Delphi Method: Techniques and Applications, Reading, MA: Addison-Wesley (1975)
  17. J.S. Armstrong, W.B. Denniston, and M.M. Gordon, 'The Use of the Decomposition Principle in Making Judgments,' Operational Behavior Human Perform, 14, pp.257-263 (1975) https://doi.org/10.1016/0030-5073(75)90028-8
  18. D.A. Seaver, D. von Winterfelt, and W. Edwards, 'Eliciting Subjective Probability Distributions on Continuous Variables,' Organizat. Behavior, Human Perform, 21, pp.379-391 (1978) https://doi.org/10.1016/0030-5073(78)90061-2
  19. R.L. Winkler, 'Cornbining Probability Distributions from Dependent Information Sour ces', Management Sciences, 27, pp.(1981)
  20. R.T. Clemen and R.L. Winkler, 'Limits fort the Precision and Value of Information from Dependent Sources,' Operational Research, 33, p.427-442 (1985)
  21. A. Mosleh and G. Apostolakis, 'The Assessment of Probability Distributions from Expert Opinions with an Application to Seismic Fragility Curves, Risk Analysis, 6(4), pp.447-461 (1986) https://doi.org/10.1111/j.1539-6924.1986.tb00957.x
  22. R.M. Cooke, 'A Theory of Weights for Combining Expert Opinion', Report 87-25, Faculty of Mathematics and Informatics/ Computer Science, Delft University of Techology, Delft, Netherlands (1987)
  23. S. Chhibber, G. Apostolakis, and D. Okrent, 'On the Use of Expert, Judgments to Estimate the Pressure Increment in the Sequoyah Containment at Vessel Breach,' Nuclear Technology, 105, pp.87-103 (1994)
  24. A. Kreuser, J. Peschke, and C. Verstegen, 'Practical Application of Expert Judgment in Assessing Operation Experience for Generating Reliability Data,' CD-Rom Proceeding of PSAM 6, San Juan, Puero Rico, USA, June 24-28 (2002)
  25. H.Glaeser, E.Hofer, M.Kloss, T.Skorek, 'Uncertainty and Sensitivity Analysis of a Post-Experiment Calculation in Thermal Hydraulics,' Reliability Engineering and System Safety, 45, pp.19-33 (1994) https://doi.org/10.1016/0951-8320(94)90073-6
  26. G. Cojazzi, L. Pinola, and R. Sardella, 'A Benchmark Exercise on Expert Judgment Techniques in PSA Level 2: Design Criteria and General Framework,' International Topical Meeting on Probabilistic Safety Assessment (PSA '96), September 29-October 3, Park City, Utah, USA, pp.1775-1778 (1996)