Confidence regions

In Bayesian statistics everything that is not known is random. That means that in the Bayesian framework statistics is simply an application of probability. Inference becomes simply a matter of finding conditional distributions. The desired unknowns are random variables – with known distributions – and what needs to be done is to find their conditional distribution given the observations.

In contrast, in frequentist statistics, some things are unknown but not random. The observations are assumed to be random, with a distribution that depends on the non-random unknowns. The objective of frequentist inference is to be able to say something interesting about the unknowns without the need to assume prior information about them.

This is done as follows: Let the sample space, i.e., the set of measurable observations be X, and let the set of potential values of the unknowns be T. Each of those may be a vector space, or a function space, or some other space. Before carrying out the experiment, every point x in X space is associated with a subset of T, say Tc(x). The association is done so that if t in T is the true value of the unknowns, then P_t(t in Tc(x)) is above some pre-determine threshold, say p = 95%.

The association Tc(x) is then called a 95% confidence region for t.

We then implicitly agree that once we carry out the experiment, we will consider only values in Tc(X) as potentially true, X being the value observed in the experiment. We are satisfied that whatever is the value of t, there is only 1-p = 5% chance that we will be mistaken.

If we want to be more careful, we set p to be closer to 1.0. If we want to allow every person to make their own p, we make a family of mappings (i.e., a family of confidence regions) with a member of the family corresponding to any value p in [0,1]. Anyone can then select the confidence region that corresponds to the p they find appropriate.

The confidence region condition is not very restrictive, and so there can be many different confidence regions for the same setting (and with the same p).

If the unknown is a real scalar, the confidence region may be a confidence interval, i.e., the mapping may be such that Tc(x) is always an interval, for any x in X. Even with this added condition, many different confidence intervals are possible.