Optimal Bayesian design by inhomogeneous Markov chain simulation

We consider decision problems defined by a utility function and an underlying probability model for all unknowns. The utility function quantifies the decision maker's preferences over consequences. The optimal decision maximizes the expected utility function where the expectation is taken with respect to all unknowns, that is, future data and parameters. In many problems, the solution is not analytically tractable. For example, the utility function might involve moments that can be computed only by numerical integration or simulation. Also, the nature of the decision space (i.e., the set of all possible actions) might have a shape or dimension that complicates the maximization. The motivating application for this discussion is the choice of a monitoring network when the optimization is performed over the high-dimensional set of all possible locations of monitoring stations, possibly including choice of the number of locations. We propose an approach to optimal Bayesian design based on inhomogeneous Markov chain simulation. We define a chain such that the limiting distribution identifies the optimal solution. The approach is closely related to simulated annealing. Standard simulated annealing algorithms assume that the target function can be evaluated for any given choice of the variable with respect to which we wish to optimize. For optimal design problems the target function (i.e., expected utility) is in general not available for efficient evaluation and might require numerical integration. We overcome the problem by defining an inhomogeneous Markov chain on an appropriately augmented space. The proposed inhomogeneous Markov chain Monte Carlo method addresses within one simulation both problems, evaluation of the expected utility and maximization.