Browse > Article

Comparison of confidence intervals for testing probabilities of a system  

Hwang, Ik-Soon (Software-Networks Department, TELECOM SudParis)
Publication Information
The Journal of the Korea institute of electronic communication sciences / v.5, no.5, 2010 , pp. 435-443 More about this Journal
Abstract
When testing systems that incorporate probabilistic behavior, it is necessary to apply test inputs a number of times in order to give a test verdict. Interval estimation can be used to assert the correctness of probabilities where the selection of confidence interval is one of the important issues for quality of testing. The Wald interval has been widely accepted for interval estimation. In this paper, we compare the Wald interval and the Agresti-Coull interval for various sizes of samples. The comparison is carried out based on the test pass probability of correct implementations and the test fail probability of incorrect implementations when these confidence intervals are used for probability testing. We consider two-sided confidence intervals to check if the probability is close to a given value. Also one-sided confidence intervals are considered in the comparison in order to check if the probability is not less than a given value. When testing probabilities using two-sided confidence intervals, we recommend the Agresti-Coull interval. For one-sided confidence intervals, the Agresti-Coull interval is recommended when the size of samples is large while either one of two confidence intervals can be used for small size samples.
Keywords
Probability testing; Interval estimation; Confidence interval; Wald Interval; Agresti-Coull interval;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 O. Markowitch, Y. Roggeman. Probabilistic non-repudiation without trusted third party. Second Conference on Security in Communication Networks 1999, Amalfi, Italy.
2 S.M. Ross, Introduction to Probability and Statistics for Engineers and Scientists, Johns Wilely & Sons, New York, 1987.
3 L. D. Brown, T. T. Cai, A. DasGupta. "Interval Estimation for a Binomial Proportion," Statistical Science, Vol. 16, No. 2, pp. 101-117, 2001.
4 T. T. Cai, "One-sided confidence intervals in discrete distributions", Journal of Statistical Planning and Inference, Vol. 131, No. 1, pp. 63-88, 2005.   DOI   ScienceOn
5 A. Agresti, B. Coull, "Approximate is better than 'exact' for interval estimation of binomial proportions", The American Statistician, Vol. 52, No. 2, pp. 119-126, 1998.   DOI   ScienceOn
6 류제복, "이항신뢰구간에 대한 소고", 한국통계 학회논문집, 제16권 제5호, pp. 731-743, 2009.   과학기술학회마을
7 Iksoon Hwang, Ana Cavalli, Testing a probabilistic FSM using interval estimation, Computer Networks, Vol. 54, No. 7 pp. 1108-1125, 2010.   DOI   ScienceOn
8 황익순, 박재성, "구간 추정 기법을 이용한 이 동통신 시스템의 통화 성공률 시험 방법", Technical report, 수원대학교, 2010.
9 방송통신위원회 이용자보호국, 2009년도 통신서비스 품질 평가 결과, 2010. URL http://www.wiseuser.go.kr/quality/main.do