Robust generalized cross-validation for choosing the regularization parameter

Let f(lambda) be the regularized solution for the problem of estimating a function or vector f(0) from noisy data y(i) = L(i)f(0) + epsilon(i), i = 1,..., n, where L-i are linear functionals. A prominent method for the selection of the crucial regularization parameter. is generalized cross-validation (GCV). It is known that GCV has good asymptotic properties as n ->infinity but it may not be reliable for small or medium sized n, sometimes giving an estimate that is far too small. We propose a new robust GCV method (RGCV) which chooses. to be the minimizer of gamma V(lambda) + (1-gamma) F(lambda), where V (lambda) is the GCV function, F(lambda) is an approximate average measure of the influence of each data point on f(lambda) and gamma is an element of (0, 1) is a robustness parameter. We show that for any n, RGCV is less likely than GCV to choose a very small value of., resulting in a more robust method. We also show that RGCV has good asymptotic properties as n ->infinity for general linear operator equations with uncorrelated errors. The function EF(lambda) approximates the risk ER(lambda) for values of. that are asymptotically a bit smaller than the minimizer of ER(lambda) ( where V (lambda) may not approximate well). The 'expected' RGCV estimate is asymptotically optimal as n ->infinity with respect to the 'robust risk' gamma ER(lambda) + (1-gamma) v(lambda), where v(lambda) is the variance component of the risk, and it has the optimal decay rate with respect to ER(lambda) and stronger error criteria. The GCV and RGCV methods are compared in numerical simulations for the problem of estimating the second derivative from noisy data. The results for RGCV with n = 51 are consistent with the asymptotic results, and, for a large range of. values, RGCV is more reliable and accurate than GCV.