Binomial proportion confidence interval

In statistics, a binomial proportion confidence interval is a confidence interval for a proportion in a statistical population. It uses the proportion estimated in a statistical sample and allows for sampling error. There are several formulas for a binomial confidence interval, but all of them rely on the assumption of a binomial distribution. A simple example of a binomial distribution is the number of heads observed when a coin is flipped ten times. In general, a binomial distribution applies when an experiment is repeated a fixed number of times, each trial of the experiment has two possible outcomes (labeled arbitrarily success and failure), the probability of success is the same for each trial, and the trials are statistically independent.

There are several ways to compute a confidence interval for a binomial proportion. The normal approximation interval is the simplest formula, and the one introduced in most basic Statistics classes and textbooks. This formula, however, is based on an approximation that does not always work well. Several competing formulas are available that perform better, especially for situations with a small sample size and a proportion very close to zero or one. The choice of interval will depend on how important it is to use a simple and easy to explain interval versus the desire for better accuracy.

Normal approximation interval

The simplest and most commonly used formula for a binomial confidence interval relies on approximating the binomial distribution with a normal distribution. This approximation is justified by the central limit theorem. The formula is

${\hat {p}}\pm z_{1-\alpha /2}{\sqrt {\frac {{\hat {p}}\left(1-{\hat {p}}\right)}{n}}}$

where ${\hat {p}}$ is the proportion of successes in a Bernoulli trials process estimated from the statistical sample, $z_{1-\alpha /2}$ is the $1-\alpha /2$ percentile of a standard normal distribution, and n is the sample size.

The central limit theorem applies well to a binomial distribution, even with a sample size less than 30, as long as the proportion is not too close to 0 or 1. For very extreme probabilities, though, a sample size of 30 or more may still be inadequate. The normal approximation fails totally when the sample proportion is exactly zero or exactly one. A frequently cited rule of thumb is that the normal approximation works well as long as np > 5 and n(1 − p) > 5; see however Brown et al. 2001.

An important theoretical derivation of this confidence interval involves the inversion of a hypothesis test. Under this formulation, the confidence interval represents those values of the population parameter that would have large p-values if they were tested as a hypothesized population proportion. The collection of values, $\theta$, for which the normal approximation is valid can be represented as

$\left\{\ \theta \ {\bigg |}\ Z_{\alpha /2}\leq {\frac {{\hat {p}}-\theta }{\sqrt {{\hat {p}}\left({1-{\hat {p}}}\right)/n}}}\leq Z_{1-\alpha /2}\right\}.$

Since the test in the middle of the inequality is a Wald test, the normal approximation interval is sometimes called the Wald interval, but Pierre-Simon Laplace described it 1812 in Théorie analytique des probabilités (pag. 283).

Wilson score interval

The Wilson interval is an improvement (the actual coverage probability is closer to the nominal value) over the normal approximation interval and was first developed by Edwin Bidwell Wilson (1927).

${\frac {{\hat {p}}+{\frac {1}{2n}}z_{1-\alpha /2}^{2}\pm z_{1-\alpha /2}{\sqrt {{\frac {{\hat {p}}\left({1-{\hat {p}}}\right)}{n}}+{\frac {z_{1-\alpha /2}^{2}}{4n^{2}}}}}}{{1+{\frac {1}{n}}}z_{1-\alpha /2}^{2}}}$

This interval has good properties even for a small number of trials and/or an extreme probability. The center of the Wilson interval

${\frac {{\hat {p}}+{\frac {1}{2n}}z_{1-\alpha /2}^{2}}{{1+{\frac {1}{n}}}z_{1-\alpha /2}^{2}}}$

can be shown to be a weighted average of ${\hat {p}}=X/n$ and $1/2$, with ${\hat {p}}$ receiving greater weight as the sample size increases. For the 95% interval, the Wilson interval is nearly identical to the normal approximation interval using $\scriptstyle {\tilde {p}}\,=\,(X+2)/(n+4)$ instead of ${\hat {p}}$.

The Wilson interval can be derived as

$\left\{\ \theta \ {\bigg |}\ Z_{\alpha /2}\leq {\frac {{\hat {p}}-\theta }{\sqrt {\theta \left({1-\theta }\right)/n}}}\leq Z_{1-\alpha /2}\ \right\}.$

The test in the middle of the inequality is a score test, so the Wilson interval is sometimes called the Wilson score interval.

Clopper-Pearson interval

The Clopper-Pearson interval is an early and very common method for calculating exact binomial confidence intervals (Clopper and Pearson 1934). This method uses the cumulative probabilities of the binomial distribution. The Clopper-Pearson interval can be written as

$\left\{\ \theta \ {\Big |}\ P\left[\mathrm {Bin} \left(n;\theta \right)\leq X\right]\geq \alpha /2\right\}\ \bigcap \ \left\{\ \theta \ {\Big |}\ P\left[\mathrm {Bin} \left(n;\theta \right)\geq X\right]\geq \alpha /2\ \right\}$

where X is the number of successes observed in the sample and Bin(n; θ) is a binomial random variable with n trials and probability of success θ.

Because of a relationship between the cumulative binomial distribution and the beta distribution, the Clopper-Pearson interval is sometimes presented in an alternate format that uses percentiles from the beta distribution. The beta distribution is, in turn, related to the F-distribution so a third formulation of the Clopper-Pearson interval uses F percentiles.

The Clopper-Pearson interval is an exact interval since it is based directly on the binomial distribution rather than any approximation to the binomial distribution. This interval, however, can be conservative because of the discrete nature of the binomial distribution. For example, the true coverage rate of a 95% Clopper-Pearson interval may be well above 95%, depending on n and θ. Thus the interval may be wider than it needs to be to achieve 95% confidence. In contrast, it is worth noting that other confidence bounds may be narrower than their nominal confidence with, i.e., the Normal Approximation (or "Standard") Interval, Wilson Interval, Agresti-Coull Interval, etc, with a nominal coverage of 95% may in fact cover less than 95%.

Agresti-Coull Interval

The Agresti-Coull interval is another approximate binomial confidence interval, developed by A. Agresti and B. Coull (1998).

Given $X$ successes in $n$ trials, define

${\tilde {n}}=n+4$

and

${\tilde {p}}={\frac {X+2}{\tilde {n}}}$

Then, a confidence interval for $p$ is given by

${\tilde {p}}\pm z_{1-\alpha /2}{\sqrt {\frac {{\tilde {p}}\left(1-{\tilde {p}}\right)}{\tilde {n}}}}$

where $z_{1-\alpha /2}$ is the $1-\alpha /2$ percentile of a standard normal distribution, as before. For example, for a 95% confidence interval, let $\alpha =0.05$, so $z_{1-\alpha /2}$ = 1.96.

Comparison of different intervals

There are several research papers that compare these and other confidence intervals for the binomial proportion. A good starting point is Agresti and Coull (1998) or Ross (2003) which point out that exact methods such as the Clopper-Pearson interval may not work as well as certain approximations. But it is still used today for many studies.

Web-based calculators

There are numerous web sites that will calculate a binomial proportion confidence interval.

• Dimension Research. Confidence Interval for Proportions Calculator uses the normal approximation.
• VassarStats. Confidence Interval of a Proportion uses the Wilson score interval method.
• causaScientia. Exact Confidence Interval for a Proportion uses a Bayesian interval with an uninformative prior distribution.
• Measuring Usability: Confidence Interval for a Completion Rate Provides simultaneous computation of Wald, Adjusted-Wald (Agresti-Coull), Exact and Score Confidence Intervals.

See also

• Coverage probability
• Estimation theory
• Krichevsky–Trofimov estimator
• Rule of three

Literature

• Agresti, Alan, and Coull, Brent A. Approximate is better than 'exact' for interval estimation of binomial proportions. The American Statistician 52: 119-126, 1998.
• Lawrence D. Brown, T. Tony Cai and Anirban DasGupta, Interval Estimation for a Binomial Proportion, Statistical Science 2001, Vol. 16, No. 2, 101–133 - pdf, with comments by Alan Agresti-Brent A. Coull, George Casella, Chris Corcoran-Cyrus Metha, Malay Ghosh, Thomas J. Santner
• Clopper, C. and Pearson, E. S. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26: 404-413, 1934.
• Ross, T. D. Accurate confidence intervals for binomial proportion and Poisson rate estimation. Computers in Biology and Medicine 33: 509-531, 2003.
• Wilson, E. B. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association 22: 209-212, 1927.

External links

• Binomial and Poisson Confidence Intervals Calculator
• Confidence Interval Calculator for a Completion Rate - Binomial Confidence Interval Calculator (xls. file)
• Binomial Distribution Calculator