The effect of self-gravity on vortex instabilities in disc-planet interactions

We study the effect of disc self-gravity on instabilities associated with gaps opened by a giant Saturn mass planet in a protoplanetary disc that lead to the formation of vortices. We also study the non-linear evolution of the vortices when this kind of instability occurs in a self-gravitating disc as well as the potential effect on type III planetary migration due to angular momentum exchange via co-orbital flows. It is shown analytically and is confirmed through linear calculations that vortex-forming modes with low azimuthal mode number, m, are stabilized by the effect of self-gravity if the background structure is assumed fixed. However, the disc's self-gravity also affects the background gap surface density profile in a way that destabilizes modes with high m. Linear calculations show that the combined effect of self-gravity through its effect on the background structure and its direct effect on the linear modes shifts the most rapidly growing vortex mode to higher m. Hydrodynamic simulations of gaps opened by a Saturn mass planet show more vortices develop with increasing disc mass and therefore importance of self-gravity. For sufficiently large disc mass the vortex instability is suppressed, consistent with analytical expectations. In this case a new global instability develops instead. In the non-linear regime, we found that vortex merging is in general increasingly delayed as the disc mass increases and in some cases multiple vortices persist until the end of simulations. For massive discs in which the vortices merge, the post-merger vortex is localized in azimuth and has similar structure to a Kida-like vortex. This is unlike the case without self-gravity where vortices merge to form a larger vortex extended in azimuth. In order to study the properties of the vortex systems without the influence of the planet, we also performed a series of supplementary simulations of co-orbital Kida-like vortices. We found that self-gravity enables Kida-like vortices to execute horseshoe turns upon encountering each other. As a result, vortex merging is avoided on time-scales where it would occur without self-gravity. Thus we suggest that mutual repulsion of self-gravitating vortices in a rotating flow is responsible for the delayed vortex merging seen in the disc-planet simulations. The effect of self-gravity on vortex-induced migration in low-viscosity discs is briefly discussed. We found that when self-gravity is included and the disc mass is in the range where vortex-forming instabilities occur, the vortex-induced type III migration of Lin & Papaloizou is delayed. There are also expected to be longer periods of slow migration between the short bursts of rapid migration compared to what occurs in a simulation without self-gravity. However, the extent of induced rapid migration is unchanged and involves flow of vortex material across the gap, independent of whether or not self-gravity is included.