Ruling out unresolved binaries in five transitional disks VLT/NACO deep 2.12 and 1.75 μm narrow-band imaging

Context. The presence of unresolved binaries on sub-arsecond scales could explain the existence of optically thin inner holes or gaps in circumstellar disks, which are commonly referred to as transitional or cold disks, and it is the first scenario to check before making any other assumptions. Aims. We aim at detecting the presence of companions inside the inner hole/gap region of a sample of five well known transitional disks using spatially-resolved imaging in the near-IR with the VLT/NACO/S13 camera, which probes projected distances from the primary of typically 0.1 to 7 arcsec. The sample includes the stars DoAr 21, HD135344B (SAO206462), HR4796A, TCha, and TWHya, spanning ages of less than 1 to 10 Myr, spectral types of A0 to K7, and hole/gap outer radii of 4 to 100 AU. Methods. In order to enhance the contrast and to avoid saturation at the core of the point-spread function (PSF), we use narrow-band filters at 1.75 and 2.12 mu m. The locally optimized combination of images (LOCI) algorithm is applied for an optimal speckle noise removal and PSF subtraction, providing an increase of 0.5-1.5mag in contrast over the classic method. Results. With the proviso that we could have missed companions owing to unfavorable projections, the VLT/NACO observations rule out the presence of unresolved companions down to an inner radius of about 0 ''. 1 from the primary in all five transitional disks and with a detection limit of 2 to 5 mag in contrast. In the disk outer regions the detection limits typically reach 8 to 9 mag in contrast and 4.7 mag for TCha. Hence, the NACO images resolve part of the inner hole/gap region of all disks with the exception of TWHya, for which the inner hole is only 4 AU. The 5 sigma sensitivity profiles, together with a selected evolutionary model, allow to discard stellar companions within the inner hole/gap region of T Cha, and down to the substellar regime for HD135344B and HR 4796A. DoAr 21 is the only object from the sample of five disks for which the NACO images are sensitive enough for a detection of objects less massive than similar to 13 M-Jup that is, potential giant planets or low-mass brown dwarfs at radii larger than similar to 76 AU ( 0 ''. 63). Conclusions. These new VLT/NACO observations further constrain the origin of the inner opacity cavities to be owing to closer or lower- mass companions or other mechanisms such as giant planet formation, efficient grain growth, and photoevaporation (for DoAr 21 and HR 4796A).