An information inequality for Bayesian analysis in imaging problems

The Cramer-Rao bound on the covariance and the Fisher information matrix are widely used in inverse (or imaging) problems. Within the Bayesian framework, thousands of works have been published where these notions have been implemented without much concern as to how they were derived and what exactly they mean. These notions were conceived originally within the frequentist framework and subsequent works extended them while still using estimators and their sampling variability. In frequentist analysis, the Cramer-Rao inequality states that the inverse of the Fisher information is a lower bound on the estimation variance of any unbiased estimator. This work revisits the issue within a Bayesian framework, where the focus is on the posterior probability density function (pdf). An information matrix and a lower bound for the covariance matrix of the posterior pdf are derived that, though looking similar to Fisher and Cramer-Rao, respectively, are different and have a different interpretation: The inverse of the B information (defined here) is a lower bound on the covariance of the posterior pdf. For this result to be valid, it is required that the posterior pdf have at least continuous second derivatives and it and its first derivatives vanish at the boundary of the domain over which they are defined. These requirements are mild and are satisfied for many problems. This analysis provides justification for a method used by practitioners and also illustrates under what conditions this approximation is appropriate.

Automated summary

This paper discusses the application of covariance and Fisher information matrix in inverse problems, as well as a reexamination within the Bayesian framework, proposing a lower bound for the covariance of the posterior probability density function.