CEEC CoE

1st CEEC Community Workshop: Energy-Efficient, Fault Resilient, and Scalable Solvers for CFD Codes
2023-12-13 14:00 - 17:00 (Europe/Paris) | Bayern
Join us for our first annual community workshop! The energy consumption constraint for large-scale computing encourages scientists to revise the architecture design of hardware but also applications, algorithms, as well as the underlying working/ storage precision. The main aim is to make the computations energy-efficient (aka sustainable) and robust numerically but also in terms of fault tolerance. On the level of algorithmic solutions, we propose to utilize all provided resources wisely by exhibiting algorithms to computation overlapping but even more communication overlapping strategies. We also promote mixed-precision strategies with the aid of computer arithmetic tools like VerifiCarlo and its variable precision backend. Hence, before lowering precision, one must ensure that the simulation is numerically correct, e.g. by relying on alternative floating-point models/ rounding to pinpoint numerical bugs and to estimate the accuracy. We also work on fault tolerant and resilient algorithms, adaptivity and meshing/ mesh refinement that adapt to the heterogeneous nature of current machines. Another issue discussed in the workshop is the adaptation of adjoint-based topology optimization methods to spectral-element CFD codes. Therefore, in this workshop, we will share our approaches, lessons learnt with preliminary results, and outline perspectives for the upcoming three years of the project.
Agenda:

14:00 Opening
14:05 Niclas Jansson, KTH. “Scalable task-parallel multigrid solvers on GPUs”
14:30 Dinesh Parthasarathy, FAU. “Optimizing multigrid solvers using grammar-guided genetic programming”
14:55 Roman Iakymchuk, UmU. “A tool-driven approach toward mixed-precision and sustainable solvers for CFD codes”
15:20 Break
15:35 Eman Bagheri and Manuel Münsch, FAU. “Fault-resilient algorithms for Exascale CFD”
16:00 Adam Peplinski, KTH. “h-type adaptive mesh refinement in spectral-element solvers: Application to Nek5000 and Neko”
16:25 Eddie Wadbro and Martin Berggren, UmU. “Topology optimization in spectral-element codes”
16:50 Summary

Register for free here!

Exploring the Ultimate Regime of Turbulent Rayleigh–Bénard Convection through Unprecedented Spectral-Element Simulations
2023-11-16 00:30 - 01:30 | Bayern
Don’t miss the official presentation of the work on NEKO that is nominated for the Gordon Bell Prize!
Official Description: We detail our developments in the high-fidelity spectral-element code Neko that are essential for unprecedented large-scale direct numerical simulations of fully developed turbulence. Major innovations are modular multi-backend design enabling performance portability across a wide range of GPUs and CPUs, a GPU-optimized preconditioner with task overlapping for the pressure-Poisson equation and in-situ data compression. We carry out initial runs of Rayleigh–Bénard Convection (RBC) at extreme scale on the LUMI and Leonardo supercomputers. We show how Neko is able to strongly scale to 16,384 GPUs and obtain results that are not possible without careful consideration and optimization of the entire simulation workflow. These developments in Neko will help resolving the long-standing question regarding the ultimate regime in RBC.

Neko – a European code development effort shortlisted for the 2023 ACM Gordon Bell Prize
2023-11-15 20:30 - 21:00 | Bayern
Come listen to our own Niclas Jansson talk about his work on NEKO, which has been shortlisted for the 2023 ACM Gordon Bell Prize. We have the great fortune of being able to build on this work during the cEEC project. Learn more in his interview at the LUMI stand (booth 206)!

Precision in linear algebra solvers and its cascade impact on applications
2023-10-11 11:15 - 12:00 | Bayern
Until recent years, linear algebra solvers were predominantly operating with double precision or binary64. With the advent of AI, in particular deep learning, lower floating-point precisions formats were introduced to accommodate the need of such computations; now the spectrum is shifted to fixed point and integer arithmetic, expanding to block floating point arithmetic. This change has also impacted the linear algebra algorithm development where the concept of mixed-precision was introduced for faster but still reliable solvers. In this talk, I would like to provide a brief overview on mixed-precision algorithmic work, introduce to my own strategy of mixing precisions in a controllable way, and provide preliminary results with applications.

VPREC to analyze the precision appetites and numerical abnormalities of several proxy applications
2023-08-24 08:30 - 10:10 | Bayern
The energy consumption constraint for large-scale computing encourages scientists to revise the architecture design of hardware but also applications, algorithms, as well as the underlying working/ storage precision. We introduce an approach to address the issue of sustainable computations from the perspective of computer arithmetic tools. We employ the variable precision backend (VPREC) to identify parts of code that can benefit from smaller floating-point formats. Finally, we show preliminary results on several proxy applications.

Sustainable and Reliable Computing with Tools: Analyzing Precision Appetites of CFD Applications with VerifiCarlo
2023-06-27 16:30 - 17:00 | Bayern
The energy consumption constraint for large-scale computing encourages scientists to revise the architecture design of hardware but also applications, algorithms, as well as the underlying working/ storage precision. The main aim is to make the computing cost sustainable and apply the lagom principle (”not too much, not too little, the right amount”), especially when it comes to working/ storage precision. Thus, I will introduce an approach to address the issue of sustainable, but still reliable, computations from the perspective of computer arithmetic tools. Before lowering precision, one needs to ensure that the simulation is numerically correct, e.g. by relying on alternative floating-point models/ rounding to pinpoint numerical bugs and to estimate the accuracy. We employ VerifiCarlo and its variable precision backend to identify the parts of the code that benefit from smaller floating-point formats. Finally, we show preliminary results on proxies of CFD applications.

MS4F - Cross-Cutting Aspects of Exploiting Exascale Platforms for High-Fidelity CFD in Turbulence Research
2023-06-27 16:00 - 18:00 | Bayern
With the arrival of exascale computing, the theoretical computational performance has increased, opening up for unprecedented simulation capabilities for Computational Fluid Dynamics (CFD) applications. While offering high theoretical peak performance and high memory bandwidth, to efficiently exploit these systems, complex programming models and significant programming investments are necessary. Furthermore, most known pre and exascale systems currently planned or installed, e.g. Frontier, Leonardo and LUMI, contain a large fraction of accelerators. Thus, the challenge of porting and tuning scientific codes for these new platforms can no longer be ignored. However, established CFD codes build on years of verification and validation of their underlying numerical methods, potentially preventing a complete rewrite and rendering disruptive code changes a delicate task. Therefore, porting established codes to accelerators poses several interdisciplinary challenges, from formulating suitable numerical methods to applying sound software engineering practices to cope with disruptive code changes. The wide range of topics makes the exascale CFD transition relevant to a broader audience, extending outside the traditional fluid dynamics community. This minisymposium aims at bringing together the CFD community as a whole, from domain scientists to HPC experts, to discuss current and future challenges towards enabling exascale fluid dynamics simulations on anticipated accelerated systems.

Neko: A Modern, Portable, and Scalable Framework for High-Fidelity Computational Fluid Dynamics
2023-06-01 17:00 - 17:30 | Bayern
Recent trends and advancements including more diverse and heterogeneous hardware in High-Performance Computing are challenging scientific software developers in their pursuit of good performance and efficient numerical methods. As a result, the well-known maxim “software outlives hardware” may no longer necessarily hold true, and researchers are today forced to re-factor their codes to leverage these powerful new heterogeneous systems. We present Neko – a portable framework for high-fidelity spectral element flow simulations. Unlike prior works, Neko adopts a modern object-oriented Fortran 2008 approach, allowing multi-tier abstractions of the solver stack and facilitating various hardware backends ranging from general-purpose processors, accelerators down to exotic vector processors and Field-Programmable Gate Arrays (FPGAs) via Neko’s device abstraction layer. Focusing on the performance and accuracy of Neko, we show the first direct numerical simulation (DNS) of a Flettner rotor submerged in a turbulent boundary layer, observing excellent agreement of lift with experimental data. Using a mesh with five million spectral elements, which turns into more than a billion unique degrees of freedom, the simulation requires less than three days to complete on accelerated systems compared to weeks on traditional non-accelerated systems. Finally, we present performance measurements on a wide range of accelerated computing platforms, including the EuroHPC pre-exascale system LUMI, where Neko achieves excellent parallel efficiency for a large DNS of turbulent fluid flow using up to 80% of the entire LUMI supercomputer.

Reliable and sustainable computations: An application-driven approach
2023-05-24 14:40 - 15:00 | Bayern
In this talk, Roman Iakymchuk presents his work on accuracy and reproducibility assuring strategies for parallel iterative solvers that may not hold due to the non-associativity of floating-point operations. These strategies primarily rely on guarding every bit of result until final rounding, hence they can be costly. The energy consumption constraint for large-scale computing encourages scientists to revise the architecture design of hardware but also applications, algorithms, as well as the underlying working/ storage precision. The main aim is to make the computing cost sustainable and apply the lagom principle (”not too much, not too little, the right amount”), especially when it comes to working/ storage precision. Thus, he will introduce an approach to address the issue of sustainable, but still reliable, computations from the perspective of computer arithmetic tools. Before lowering precision, one must ensure that the simulation is numerically correct, e.g. by relying on alternative floating-point models/ rounding to pinpoint numerical bugs and to estimate the accuracy. We employ VerifiCarlo and its variable precision backend to identify the parts of the code that benefit from smaller floating-point formats. Finally, we show preliminary results on proxy applications.

Introducing CEEC at ISC23
2023-05-24 11:45 - 12:15 | Bayern
Come learn about CEEC at the European HPC Joint Undertaking booth at ISC23! We’ll give a quick overview of the project and be eager to answer all your questions about CFD, our lighthouse cases, and how our work can help address grand challenges.

ISC High Performance 2023
2023-05-21 - 2023-05-25 | Bayern
Come visit us at the EuroHPC Joint Undertaking booth B201 at ISC! We will be there all week to answer your questions about our goals, future work, and its potential impact on the field of computational fluid dynamics. Once again in Hamburg, the annual ISC event is sure to offer something for every person interested in HPC, machine learning, data analytics, and quantum computing.

"Scalable High-Fidelity Simulation of Turbulence With Neko Using Accelerators" CUG 2023 “Sustainable Exascale”
2023-05-11 14:00 - 14:30 | CSCT Center for Science Ltd (CSC Finland), Helsinki, Finland
It’s time again for the annual Cray User Group (CUG) conference: this time in Finland! We’ll be contributing to the discussions on how to best use Cray and HPE supercomputers with a paper titled, “Scalable High-Fidelity Simulation of Turbulence With Neko Using Accelerators”. You will also be able to read the paper on our publication page after the conference.
