Alya – High Performance Computational Mechanics
Technical Description
Alya is a multi-physics simulation code developed at Barcelona Supercomputing Center (BSC). From its inception, the Alya code is designed using advanced HPC programming techniques to solve coupled problems on supercomputers efficiently, especially taking heterogeneous architectures into account. The target domain are engineering applications, with all their particular features. Since its beginnings in 2004, Alya has scaled well on an increasing number of processors when solving single-physics problems such as fluid mechanics, solid mechanics, heat transfer or combustion. Over time, we have strived a concerted effort to maintain and even improve scalability for multi-physics problems. This poses challenges on multiple fronts, including numerical models, parallel implementation, physical coupling models, algorithms and solution schemes, and meshing processes.
Alya Explainer
If the above technical description was not what you’re looking for, check out the explainer answers to some common questions below.
What can Alya simulate?
Alya is a multi-physics code meaning that it doesn’t just do fluid mechanics but also solid mechanics and particle flow, among others. It is useful for simulations like the aeroelastic lighthouse case where it would simulate the mechanical stresses on the airplane structure based on the fluid flow simulation being computed by SOD2. Naturally, the fluid flow will impact and apply stress to the airplane structure just as the airplane will cause the fluid to become turbulent upon impact. Simulating the interactions of these two forces is what we mean when we say that it is a coupled simulation. It is also the second function of Alya in CEEC. In addition to simulating the solid mechanics of the airplane structure, Alya will couple and coordinate the actions and needs of the two simulations.
What are the trade-offs of using Alya?
Alya serves two functions in CEEC. It simulates the solid mechanics of the airplane and provides the code and tools for coupling this simulation with the fluid flow simulation provided by SOD2. Although the solid mechanics aspect is fairly classic and could perhaps be provided by another code, there aren’t any good options for replacing the coupling functionality of Alya. Hence, with no other real alternatives to Alya, there are no drawbacks to using it.
What could Alya improve?
That said, although Alya has been around for a long time and much of it already runs on GPUs, the solid mechanics part of Alya isn’t yet optimized to run on GPUs, which means that it’s not running as quickly as it could on newer HPC systems.
What will CEEC aim to improve about Alya?
Porting the solid mechanics to GPUs might seem obvious, but when this simulation is being coupled with another simulation that is already using the GPUs, it’s not clear how or if porting the solid mechanics to GPUs would actually work. Instead, CEEC will work to improve the solid mechanics functionality of Alya in two other ways.
First, we will improve the iterative solvers used during the simulation because the convergence of iterative solvers in solid mechanics is known to be relatively poor. Second, if the solid mechanics functionality stays on CPUs, we will improve the node level performance of the code to make sure it’s using all possible new optimizations to run as efficiently as possible.
In any case, we will work to improve dynamic load balancing (DLB) to ensure that each simulation can make the best possible use of the hardware available at any given moment during simulation runtime. This may mean that we need to ensure the solid mechanics can use CPUs inside the nodes where SOC2 is running its fluid flow on GPUs. SOD2 will be using some of that CPU power to “manage” work being done by GPUs using message passing interface (MPI) processes, however, the rest of the computational power could potentially be used by Alya’s solid mechanics simulation.
Why are these improvements a challenge?
Alya is based on a classical finite element method using Lagrange interpolation functions while SOD2D uses spectral finite elements. Thus, the improvements are challenging because we don’t yet know how coupling solid mechanics from Alya and fluid mechanics from SOD2D will work at a larger scale with such different interpolations. For Alya, many of the easiest or most obvious optimizations have already been made. Furthermore, not knowing whether or not the solid mechanics can be ported to running on GPUs along side a fluid mechanics simulation also using GPUs makes it hard to know what kinds of optimizations will even be needed for the solid mechanics functionality. Will we need to focus on parallelization and load balancing across GPUs or node level performance on CPUs?
Again, when optimizing the coupling itself, we don’t know how the algorithms for classical finite element solid mechanics and spectral element fluid mechanics will interact. There will certainly be opportunities to make the coupling run more efficiently and reliably, but not knowing what kind of problems and where those problems might occur makes solving them much more challenging.
Ultimately, the challenge is to make all of these improvements by finding unknown solutions to currently unknown problems.
What real-world benefits will an improved Alya help achieve?
The truly impactful part of this work will be having the ability to couple fluid flow with solid mechanics at a larger scale. Knowing not just how the fluid flows around an object, but also how the object moves in response, and how those movements subsequently interact with each other provides a whole new level of realism and detail to work with. All the same familiar impacts of improved CFD simulation would be magnified and lead to accelerated improvement in structural design for energy savings, sustainability, and safety.