Sod2D – Spectral high-Order coDe 2 solve partial Differential equations
Technical Description
Sod2 implements a numerical solution for the equations governing compressible and incompressible fluid flow in three dimensions. The code is based on the spectral element method (SEM) and is designed to be used for scale-resolving simulations (LES and DNS). It is written in Fortran, and uses MPI and OpenACC to provide parallelism at both coarse and fine-grained levels. The mesh is partitioned using the GEMPA library , which is included as a submodule in this repository. It also uses HDF5 for I/O, which must be installed appropriately according to the desired platform.
SOD2 Explainer
If the above technical description was not what you’re looking for, check out the explainer answers to some common questions below.
What can SOD2D simulate?
SOD2D is exclusively for simulating fluid flow. It focuses on compressible flow including compressible flow at higher speeds like those used for simulating airplane flight. This is why it will be part of the codes used for the Aeroelastic simulation lighthouse case. SOD2D will simulate the movement and behavior of air around the airplane while Alya simulates the movement and behavior of the airplane itself when moving through the fluid flow. Naturally, the fluid flow will impact and apply stress to the airplane structure just as the airplane will cause the fluid to become turbulent upon impact. Simulating the interactions of these two forces is what we mean when we say that it is a coupled simulation.
The only other CEEC code capable of higher speed compressible flows is FLEXI, which we are using in our only other airplane related lighthouse case on shock boundary layers over an airplane wing.
What are the trade-offs of using SOD2D?
The largest advantage to using SOD2D is that it uses the spectral element method but nevertheless can simulate flow around complex geometries like an entire airplane. This puts it ahead of many codes both inside and outside of CEEC. As an example, Neko can’t currently simulate flow around complex geometries while using hexahedral grid elements (building blocks) to build the flow simulation, but SOD2D can already do this despite also using hexahedral elements by exploiting the splitting of high order tetrahedral meshes.
SOD2D also can do either direct numerical simulation (DNS) like Neko and Nek but can also use turbulence modeling like Nek and FLEXI.
All of this being said, SOD2D isn’t yet as scalable as some other codes and definitely not as scalable as it could be.
What could SOD2D do better?
SOD2D has room to improve its scalability. It has been tested with hundreds of GPUs but will need to run on thousands or hundreds of thousands of GPUs to make full use of upcoming Exascale machines.
What will CEEC aim to improve about SOD2D?
Moving from hundreds to hundreds of thousands of GPUs is unrealistic for 4 years. Hence, the goal during CEEC is to move from hundreds of GPUs to the next intermediate step in scalability, thousands of GPUs.
We also will work to improve SOD2D’s time to solution, not how much it uses the computational potential of available hardware, but how quickly it finishes using the hardware and provides results of the simulation.
Lastly, we will integrate machine learning models to help optimize turbulence models. Currently turbulence models are built using physical theories, but they could also be built using experimental data or data from direct numerical simulations. Hence, we will use the direct numerical simulation data produced by SOD2D to train better turbulence models for SOD2D to use.
Why are these improvements a challenge?
Overall, everything we are trying to improve is exponentially more complicated because we want these improvements to be maintained while coupling SOD2D with another simulation and another code framework, in this case solid mechanics and Alya. Although optimizing fluid mechanics codes has been done a great deal, there is far less knowledge about how to optimize coupled fluid and solid dynamics codes and simulations. CEEC will be working on the unknown unknowns in order to achieve our planned improvements.
What real-world benefits will an improved SOD2D help achieve?
Improving SOD2D alone will confer many of the same benefits of improving other codes within CEEC. It will allow us to run more accurate simulations of fluid flow, simulate how they are impacted by more realistic geometrical objects, and thus provide more opportunities for applying this knowledge to real-world solutions like better, quieter, more efficient transportation. Developing machine learning powered turbulence models will also improve our overall understanding of turbulence and ability to model it in SOD2D and other codes.
However, the truly impactful part of this work will be having the ability to couple fluid flow with solid mechanics at a larger scale. Knowing not just how the fluid flows around an object, but also how the object moves in response, and how those movements subsequently interact with each other provides a whole new level of realism and detail to work with. All the same familiar impacts of improved CFD simulation would be magnified and lead to accelerated improvement in structural design for energy savings, sustainability, and safety.