Accurately simulating atmospheric boundary layer flows is challenging because of their complex, multi-scale turbulence, limiting the reliability of wind predictions used in applications such as wind-energy planning and turbine design. CEEC used extreme-scale computing to perform high-resolution simulations with up to 100 billion grid points, providing more detailed atmospheric-flow information that could support improved wind-resource assessment, AEP modelling, and turbine-load analysis.
| Industry | Scientific Domaine | Code |
| Wind Energy | CFD / Atmospheric Science | Nek5000/NekRS |
Description of the Challenge
In addition to vertically exchanging moisture and aerosols in the atmosphere, Atmospheric Boundary Layer (ABL) flows affect transportation, renewable power generation (wind and solar), and pollutant dispersion, among others. Thus, being able to simulate ABL flow is important for the study of wind farms, urban canyons, and basic weather modeling. However, these highly turbulent flows are impacted by density stratification from surface heating and cooling, regional weather patterns, terrain morphology, and Coriolis effects due to the Earth’s rotation.
Until now, researchers have focused on developing reliable high-fidelity Large-Eddy-Simulation (LES) models for ABL flows and improved wall models. These developments enable more detailed simulations of atmospheric flows, but the reliability of the results also depends on how the models are implemented and configured.
For this reason, there is a need to systematically assess the sensitivity of these models to mesh resolution, numerical discretization, SGS parameterizations, and surface boundary conditions. This helps determine which modelling choices have the greatest influence on the results and what is required to obtain reliable predictions.
The two examined benchmark cases in CEEC, GABLS (GEWEX Atmospheric Boundary Layer Study) and CUABL (Convectively Unstable Atmospheric Boundary Layer), focus on stable and unstable atmospheric boundary layer flows, respectively, investigating Reynolds numbers over 50M and considering computational domains that extend over several hundreds of meters in scale to capture the relevant turbulence structures and stratification effects, using extreme scale resolution.
Running simulations at this size and resolution requires access to extreme-scale computing resources and substantial numbers of node hours. CEEC brings the expertise needed to run these large-scale simulations efficiently and to systematically compare the results obtained with different modelling approaches and configurations. This makes it possible to assess the sensitivity of the simulations and identify the modelling choices required for reliable ABL predictions.
Why this Matters for Industry
Accurate high-resolution LES of stable and convective ABL enables detailed representation of intermittent turbulence, coherent flow structures, and stability-dependent shear effects, all of which are important for accurately modelling a variety of practical applications. For example, improved ABL modelling has potential relevance to urban modelling, weather prediction, aviation, wildfire prediction, and wind energy.
For this story, we will focus on how these specific flow attributes are key drivers of unsteady wind turbine loading. By capturing these processes more faithfully, high-resolution LES can provide more detailed predictions of both mean flow characteristics and higher-order turbulence statistics, including wind speed distributions, shear profiles, and temporal variability across different atmospheric stability regimes. This, in turn, can support more accurate wind resource assessment and provide a stronger basis for Annual Energy Production (AEP) predictions by providing more detailed information about the underlying wind conditions. Furthermore, a more accurate characterization of intermittent and unsteady wind conditions can provide valuable input for subsequent analyses of structural fatigue and lifetime and for developing advanced control strategies aimed at mitigating mechanical stress and optimizing turbine performance.
▶ Scientific Background
Our work on this Lighthouse Case has advanced the state of the art in LES of stable and convective ABL by exploiting extreme-scale computing resources to reach unprecedented mesh resolutions that would be impractical for a systematic study at this scale without access to such resources.
The work provides a comprehensive assessment of LES solution quality, focusing on sensitivities to grid resolution, subgrid-scale (SGS) parameterizations, numerical discretization approaches, and surface boundary conditions. In other words, the study examines how strongly the simulation results depend on the choices made when setting up and configuring the model. This helps identify which modelling choices are most important for obtaining reliable ABL predictions at high resolution.
In addition, it has extensively investigated best practices in two well-known benchmark problems in extreme-scale simulation of ABL flows, delivering new modelling configurations and strategies, including enhanced wall and SGS modeling strategies. These configurations make effective use of existing Nek5000 and NekRS capabilities for the demanding ABL simulations.
▶ Technical Details: Exascale Computing Approach
Extreme-scale computing resources are required for this study because of the inherently multi-scale and computationally demanding nature of high-resolution Large Eddy Simulations (LES) of the atmospheric boundary layer (ABL). Accurately resolving both stable and convective regimes necessitates capturing a wide spectrum of turbulent motions, ranging from large energy-containing eddies down to smaller scales that are normally represented by subgrid-scale (SGS) models, particularly under stable stratification where turbulence structures become finer and more intermittent.
Stable stratification occurs when atmospheric conditions suppress vertical motion and mixing, for example during nighttime cooling of the Earth’s surface. Under these conditions, turbulence can become weaker, finer and more intermittent than in convective conditions, such as during daytime surface heating.
Capturing a bigger part of the spectrum of turbulent scales allows the systematic assessment of sensitivity to mesh resolution, numerical discretization, SGS parameterizations, and surface boundary conditions. Finer meshes allow a more direct representation of smaller turbulent eddies, reducing the dependence on SGS models to represent these scales and providing results that are more representative of the physical flow. For this reason, extremely fine grids of up to 100B total number of grid points were utilized for the two benchmarks.
Furthermore, meaningful analysis requires time integration for long physical times, of several hours, to ensure statistical convergence of both low- and high-order flow quantities, especially for cases such as the nocturnal, stable ABL or the sheared convective boundary layer where transient effects and slow-evolving processes, like mixing and entrainment are critical. This requires millions of time steps and allows the simulations to capture the statistical behaviour of the flow over sufficiently long periods.
The runtime for the integration to such physical times, corresponding to millions of timesteps, would be prohibitively long on non-exascale systems. In addition, the study requires repeated simulations at different resolutions and configurations in order to systematically compare their results. Access to extreme-scale computing resources therefore makes the overall study feasible within a practical timeframe. In addition, the use of high-order methods in Nek5000 and NekRS, along with advanced wall models that account for near-surface atmospheric effects, further increases computational intensity while improving accuracy, and results in excellent strong scaling properties at exascale compared to low-order methods.
Extensive scaling studies, code comparisons, and cross-validation efforts across CPU and GPU architectures had to be performed and demanded repeated simulations at varying resolutions and configurations. Together, these factors make exascale computing essential to achieve the required fidelity, throughput, and turnaround time, enabling both detailed physical insight and robust model validation. Additionally, the high-resolution ABL simulations with NekRS offer an excellent foundation for future work that can couple it with other codes, such as MPTRAC and ICON, which are used for meso- and macroscale Earth system modeling.
Results and New Insights
The high-resolution simulations of the GABLS and CUABL benchmark cases provide detailed insight into the turbulent structure of both stable and convective atmospheric boundary layer regimes, while also providing increasingly converged predictions of key bulk flow quantities.
Key overall flow characteristics include mean wind velocity and temperature profiles and their variability.
Key findings include:
- Convergence of mean profiles and variances of wind velocity and temperature for the ABL flows, indicating reliable prediction of bulk atmospheric behavior across stability regimes
- Clear identification of the size, structure, and orientation of vortical features, and how these are strongly influenced by atmospheric stability conditions
- Resolution of smaller, dynamically active eddies as grid resolution increases, highlighting the importance of fine-scale turbulence in accurately capturing flow physics
Taken together, these results highlight that high-fidelity LES of ABL flows not only provides a more detailed understanding of fundamental turbulence dynamics across stability regimes but also provides detailed atmospheric-flow information that could support industrially relevant applications such as wind energy resource assessment and turbine-load analysis, as well as environmental flow modelling.
Industrial Takeaways
The high-resolution LES results provide detailed atmospheric-flow information that could be used in several important industrial applications, particularly for wind energy and environmental modelling.
For wind-energy applications, the improved representation of turbulence, shear, atmospheric stability and temporal variability can provide more detailed input for wind-resource assessment and Annual Energy Production (AEP) modelling. The systematic assessment of resolution and modelling choices also helps identify the level of computational detail needed to obtain reliable results, providing a basis for balancing simulation fidelity against computational cost.
The enhanced characterization of intermittent turbulence and unsteady wind conditions could also provide more detailed input for subsequent analyses of turbine loading, structural fatigue and lifetime, as well as for the development of advanced turbine control strategies.
Beyond wind energy, the improved representation of turbulent atmospheric structures could support future applications involving pollutant dispersion and microclimate effects. For these applications, resolving smaller, dynamically active eddies can provide more detailed information about the turbulent structures that influence how air, heat and other transported quantities move through the atmosphere. The systematic assessment of resolution and modelling choices can also help establish which configurations are needed to obtain reliable results for a given application.
Altogether, these advances provide a strong foundation for integrating high-fidelity simulation tools into industrial workflows, where the resulting atmospheric-flow information can be used as input to more predictive, efficient, and better-informed design and analysis strategies.
▶ Codes and Software Stack
Nek5000 & NekRS are highly-efficient and scalable open source incompressible and low Mach flow solvers employing the spectral element method (SEM), a high-order weighted residual technique for spatial discretization. The principal advantage of the SEM is that they require significantly fewer grid points per wavelength in order to accurately propagate a signal (or turbulent structure) over extended times in high Reynolds number simulations, such as those of ABL flows. In this way, higher accuracy can be achieved, and finer structures can be captured with lower computational cost compared to low-order discretization methods.
The Nek ecosystem has established state-of-the-art performance at scale, with Nek5000 delivering highly efficient CPU-based simulations, and NekRS achieving excellent scalability across both GPU and CPU architectures on flagship HPC systems, including Summit, Aurora, Polaris, JUWELS, LUMI, Leonardo, and MareNostrum 5.
The open-source nature of the codes allows further development and potential adoption by both academic and industrial users also in cases where a specific level of accuracy is desired, in which the employment of SEM allows for energy and time efficiency.