A european southern observatory very large telescope survey of near-infrared (Z≤25) selected galaxies at redshifts 4.5<z<6:: Constraining the cosmic star formation rate near the reionization epoch

We present the results of a Very Large Telescope (VLT) and Hubble Space Telescope imaging survey aimed at the identification of 4.5 < z < 6 galaxies. In the VLT data, a set of broadband and intermediate-band filters has been used to select 13 high-z candidates in a Z(AB) less than or equal to 25 mag catalog, over an area of about 30 arcmin(2). Discrimination against lower redshift interlopers ( mainly early-type galaxies at high redshift and cool Galactic stars) has been done combining morphological and spectral classification. This sample has been combined with a deeper I-AB less than or equal to 27.2 mag sample obtained from the Hubble Deep Field (HDF) campaigns. The VLT final sample consists of 13 high-z candidates, four of which are identified with high confidence as z > 4: 5 galaxies. The resulting integral surface density of the Z(AB) < 25 candidates at z > 4.5 is in the range 0.13-0.44 arcmin(-2), and that in the highest redshift bin 5 < z <= 6 is between 0.07 and 0.13 arcmin(-2). In the two HDFs, we identify 25 galaxies at I-AB <= 27.2 in the range 4.5 <= z < 5 and 16 at 5 less than or equal to z less than or equal to 6, corresponding to surface densities of 3.1 and 2 arcmin(-2), respectively. We show that the observed Z(AB) < 25 UV luminosity density (LD) appears to drop by about 1 order of magnitude from z similar or equal to 3 to 6. However, if we apply a threshold to obtain an absolute magnitude-limited sample, the UV LD is roughly constant up to z similar or equal to 6. We finally show that recent semianalytic hierarchical models for galaxy formation, while predicting a nearly constant total UV LD up to z similar or equal to 6, underpredict the observed UV LD at Z(AB) similar or equal to 25 and overpredict the I-AB <= 27.2 one. This behavior can be understood in terms of a poor match to the slope of the UV luminosity function.