PaScaL_TCS: A versatile solver for large-scale turbulent convective heat transfer problems with temperature-dependent fluid properties

PaScaL_TCS is an efficient and scalable solver for large-scale direct numerical simulations of natural convective flow that considers temperature-dependent fluid properties. For increased scalability on a massive-scale distributed memory system, the solver decomposes the computational domain into a cubic sub-domain. The numerical procedure is based on the monolithic projection method with staggered time discretization [1], which is an implicit and non-iterative solver for wall-bounded domains. The PaScaL_TDMA library is used to solve batched tridiagonal systems, which are partitioned according to domain decomposition. For parallel fast Fourier transform in a Fourier Poisson solver for pressure, two transpose schemes with different communicator sizes are proposed and compared according to the number of cores. An explicit intermediate aggregation scheme for MPI-IO is suggested to reduce the number of processes that simultaneously take part in parallel IO. The overall implementation was evaluated using the Nurion cluster system of the Korea Institute of Science and Technology Information with detailed performance profiling. With the efficient communication, the results showed good strong and weak scalability up to 131,072 cores. The file IO performance improved with the suggested MPI-IO scheme using explicit aggregation, especially when many processes are involved in parallel IO. The solver was verified by conducting large-scale differentially heated vertical convection flow simulations under the Oberbeck-Boussinesq (OB) approximation. The capability of the solver to capture and quantitatively describe the non-OB effect was demonstrated via glycerol Rayleigh-Benard convection flow simulations.Program summaryProgram Title: PaScaL_TCSCPC Library link to program files: https://doi .org /10 .17632 /3b2bysrcxm .1Developer's repository link: https://github .com /MPMC-LabLicensing provisions: MITProgramming language: Fortran90Nature of problem: Solving the incompressible Navier-Stokes equations for natural convective flow considering the non-Oberbeck-Boussinesq effect, which considers the temperature dependence of fluids properties, in three-dimensional channel domain.Solution method: PaScaL_TCS uses the monolithic projection-based method with staggered time discretization (MPM-STD) [1]. It employs the Crank-Nicolson scheme along with staggered time which evaluates scalar and vector variables at the (n +1/2) and (n +1) time level, respectively. PaScaL_TDMA [2] is used to solve multiple tridiagonal matrix systems for decoupled energy and momentum equations in a distributed memory system. The Poisson equation solver for the pressure-correction scheme is based on Fourier diagonalization. MPI_Alltoallw is used for parallel FFT. PaScaL_TCS supports a single-file-based parallel IO for efficient data generation in large-scale computing.& COPY; 2023 Elsevier B.V. All rights reserved.

Automated summary

PaScaL_TCS is an efficient and scalable solver for large-scale direct numerical simulations of natural convective flow that considers temperature-dependent fluid properties. The solver decomposes the computational domain into a cubic sub-domain for increased scalability on a massive-scale distributed memory system. It uses the monolithic projection method with staggered time discretization and solves batched tridiagonal systems using the PaScaL_TDMA library. Two transpose schemes with different communicator sizes are proposed and compared for parallel fast Fourier transform. An explicit intermediate aggregation scheme for MPI-IO is suggested to reduce the number of processes involved in parallel IO. The solver demonstrates good strong and weak scalability up to 131,072 cores and is verified through simulations of differentially heated vertical convection flow and glycerol Rayleigh-Benard convection flow.