Bayes estimator

In estimation theory and decision theory, a Bayes estimator or a Bayes action is an estimator or decision rule that minimizes the posterior expected value of a loss function (i.e., the posterior expected loss). Equivalently, it maximizes the posterior expectation of a utility function. An alternative way of formulating an estimator within Bayesian statistics is maximum a posteriori estimation.

Definition[edit]

Suppose an unknown parameter $\theta$ is known to have a prior distribution $\pi$ . Let ${\widehat {\theta }}={\widehat {\theta }}(x)$ be an estimator of $\theta$ (based on some measurements x), and let $L(\theta ,{\widehat {\theta }})$ be a loss function, such as squared error. The Bayes risk of ${\widehat {\theta }}$ is defined as $E_{\pi }(L(\theta ,{\widehat {\theta }}))$ , where the expectation is taken over the probability distribution of $\theta$ : this defines the risk function as a function of ${\widehat {\theta }}$ . An estimator ${\widehat {\theta }}$ is said to be a Bayes estimator if it minimizes the Bayes risk among all estimators. Equivalently, the estimator which minimizes the posterior expected loss $E(L(\theta ,{\widehat {\theta }})|x)$ for each $x$ also minimizes the Bayes risk and therefore is a Bayes estimator.^[1]

If the prior is improper then an estimator which minimizes the posterior expected loss for each $x$ is called a generalized Bayes estimator.^[2]

If $x|\theta$ is , $x|\theta \sim N(\theta ,\sigma ^{2})$ , and the prior is normal, $\theta \sim N(\mu ,\tau ^{2})$ , then the posterior is also Normal and the Bayes estimator under MSE is given by

Normal

If a Bayes rule is unique then it is admissible. For example, as stated above, under mean squared error (MSE) the Bayes rule is unique and therefore admissible.

[5]

If θ belongs to a , then all Bayes rules are admissible.

discrete set

If θ belongs to a continuous (non-discrete) set, and if the risk function R(θ,δ) is continuous in θ for every δ, then all Bayes rules are admissible.

Recursive Bayesian estimation

Generalized expected utility

Lehmann, E. L.; Casella, G. (1998). Theory of Point Estimation (2nd ed.). Springer. 0-387-98502-6.

ISBN

(1985). Statistical decision theory and Bayesian Analysis (2nd ed.). New York: Springer-Verlag. ISBN 0-387-96098-8. MR 0804611.

Berger, James O.

, Encyclopedia of Mathematics, EMS Press, 2001 [1994]