Phonon mode calculations and Raman spectroscopy of the bulk-insulating topological insulator BiSbTeSe2

The tetradymite compound BiSbTeSe2 is one of the most bulk-insulating three-dimensional topological insulators, which makes it important in topological insulator research. It is a member of the solid-solution system Bi2-xSbxTe3-ySey, for which the local crystal structure, such as the occupation probabilities of each atomic site, is not well understood. We have investigated the temperature- and polarization-dependent spontaneous Raman scattering in BiSbTeSe2, revealing a much higher number of lattice vibrational modes than predicted by group-theoretical considerations for the space group R (3) over barm corresponding to an ideally random solid-solution situation. The density-functional calculations of phonon frequencies show a very good agreement with experimental data for parent material Bi2Te3, where no disorder effects were found. In comparison to Bi2Te3 the stacking disorder in BiSbTeSe2 causes a discrepancy between theory and experiment. Combined analysis of experimental Raman spectra and density-functional-theory-calculated phonon spectra for different types of atomic orders showed coexistence of different sequences of layers in the material and that those with Se in the center and a local order of Se-Bi-Se-Sb-Te are the most favored.