Squared visibility estimators: Calibrating biases to reach very high dynamic range

In the near-infrared, where detectors are limited by readout noise, most interferometers have been operated in wide band in order to benefit from larger photon rates. We analyze in this paper the biases caused by instrumental and turbulent effects to V-2 estimators for both narrowband and wideband cases. Visibilities are estimated from samples of the interferogram using two different estimators, V-1(2), which is the classical sum of the squared modulus of Fourier components, and a new estimator V-2(2), for which complex Fourier components are summed prior to taking the square. We present an approach for systematically evaluating the performance and limits of each estimator and for optimizing observing parameters for each. We include the effects of spectral bandwidth, chromatic dispersion, scan length, and differential piston. We also establish the expression of the signal-to-noise ratio of the two estimators with respect to detector and photon noise. The V-1(2) estimator is insensitive to dispersion and is always more sensitive than the V-2(2) estimator. However, the latter allows us to reach better accuracies when detection is differential piston noise limited. Biases and noise directly impact the dynamic range of reconstructed images. Very high dynamic ranges are required for direct exoplanet detection by interferometric techniques, thus requiring estimators to be bias-free or biases to be accurately calibrated. We discuss which estimator and which conditions are optimum for astronomical applications, especially when high-accuracy visibilities are required. We show that there is no theoretical limit to measuring visibilities with accuracies as good as 10(-5), which is important in the prospect of detecting faint exoplanets with interferometers.