Workflows are the New Applications: Challenges in Performance, Portability, and Productivity

Great strides have been made to enable performance, portability, and productivity in HPC, but the focus has so far been on standalone applications and on-node programming models. Complex workflows comprising many communicating orchestrators, services, and simulations are redefining what modern HPC applications look like, and it is time to revisit what performance, portability, and performance mean when considered at the workflow level. What definitions make the most sense for workflows? How can workflows be deployed across HPC centers with incompatible software? And how can we ensure that not just one but many applications are deployed reliably and in an optimized way across different clusters? This paper highlights trends, reveals new challenges, and points to some potential solutions.