Instabilities of vortices in a binary mixture of trapped Bose-Einstein condensates: Role of collective excitations with positive and negative energies

Correspondence between frequency and energy spectra and biorthogonality conditions for the excitations of Bose-Einstein condensates described by the Gross-Pitaevskii model have been derived self-consistently using general properties arising from the Hamiltonian structure of the model. It has been demonstrated that frequency resonances of the excitations with positive and negative energies can lead to their mutual annihilation and appearance of the collective modes with complex frequencies and zero energies. Conditions for the avoided crossing of energy levels have also been discussed. General theory has been verified both numerically and analytically in the weak interaction limit considering an example of vortices in a binary mixture of condensates. Growth of excitations with complex frequencies lends to spiraling of the unit and double vortices out of the condensate center to its periphery and to splitting of the double- and higher order vortices to the unit-order vortices.