Shift and broadening in attenuated total reflection spectra of the hyperfine-structure-resolved D2 line of dense rubidium vapor

We have observed attenuated total reflection at a glass/rubidium vapor interface in high precision. The atom density was from 3.8x10(20) to 3.7x10(21) m(-3) and the angle of incidence was from 48.4 to 64.8 degrees. Redshift and broadening of the reflection spectra were observed. The redshift depended on atom density, angle of incidence, light polarization and hyperfine structure. The observed broadening was attributed to the self-broadening of the resonance line. Calculation based on the Fresnel formula with a local index for the susceptibility of the vapor, which included the Lorentz local field correction and the modification of the wave vector of the evanescent light by the optically dense atomic vapor, was found to be in reasonable agreement with the experimental results.