High-performance P3M N-body code: CUBEP3M

This paper presents (CUBEPM)-M-3, a publicly available high-performance cosmological N-body code and describes many utilities and extensions that have been added to the standard package. These include a memory-light runtime spherical overdensity halo finder, a non-Gaussian initial conditions generator and a system of unique particle identification. (CUBEPM)-M-3 is fast, its accuracy is tuneable to optimize speed or memory and has been run on more than 27 000 cores, achieving within a factor of 2 of ideal weak scaling even at this problem size. The code can be run in an extra-lean mode where the peak memory imprint for large runs is as low as 37 bytes per particles, which is almost two times leaner than other widely used N-body codes. However, load imbalances can increase this requirement by a factor of 2, such that fast configurations with all the utilities enabled and load imbalances factored in require between 70 and 120 bytes per particles. (CUBEPM)-M-3 is well designed to study large-scale cosmological systems, where imbalances are not too large and adaptive time-stepping not essential. It has already been used for a broad number of science applications that require either large samples of non-linear realizations or very large darkmatter N-body simulations, including cosmological reionization, halo formation, baryonic acoustic oscillations, weak lensing or non-Gaussian statistics. We discuss the structure, the accuracy, known systematic effects and the scaling performance of the code and its utilities, when applicable.