At A Glance
| Code / tool | Neko-based simulation workflow; ADIOS2; PySEMTools |
| Technical area | GPU-resident optimisation; parallel scalability; topology optimisation |
| Challenge | Managing the large time history required by adjoint simulations without excessive storage, I/O and recomputation |
| CEEC contribution | Integrated streaming POD compression into the forward simulation workflow and used reconstructed states in the adjoint calculation |
| Key achievement | 29–31% runtime reduction in two 100-iteration optimisation cases; final objective values within 1.5% of checkpointing references |
| Impact | Lower data-management overhead for time-dependent topology optimisation on GPU-accelerated systems |
The Challenge
Topology optimisation of unsteady fluid flows requires repeated forward and adjoint simulations. The forward simulation calculates how the flow evolves over time, while the adjoint simulation runs backwards through that time history to determine how changes in the design affect the optimisation objective.
The adjoint calculation needs access to the forward flow state at every time step. For high-resolution simulations with many time steps, retaining the complete flow history can require an impractical amount of storage. Alternatively, the simulation states can be stored at selected intervals and recomputed when needed. This approach, known as checkpointing, reduces storage requirements but introduces additional file input/output (I/O) and computational overhead.
These costs accumulate over an optimisation, which may require hundreds of iterations. Efficiently managing the forward flow history is therefore an important part of making high-fidelity topology optimisation practical on GPU-accelerated supercomputers.
The CEEC Solution
CEEC developed an alternative to checkpointing based on in-situ Proper Orthogonal Decomposition (POD), which compresses the forward flow history while the simulation is running. POD represents a time-dependent flow field using a reduced set of spatial modes and their time-dependent coefficients. Instead of retaining every full flow state, the method stores a compact representation that can be used to reconstruct the states needed during the adjoint calculation.
The POD implementation integrates several components into the simulation workflow:
- GPU-based flow simulation with Neko: The forward simulation runs on the GPUs of LUMI-G.
- Parallel POD compression with PySEMTools (also maintained by CEEC): The streaming POD implementation runs on CPU cores alongside the GPU computation, making use of resources that would otherwise be underused.
- In-situ data streaming with ADIOS2 acting as a data transfer layer between Neko and PySEMTools: Flow data are streamed between the simulation and streaming POD implementation .
- Reconstruction for the adjoint calculation: The compressed flow history is used to reconstruct the states needed for adjoint sensitivity analysis, reducing dependence on checkpoint I/O and recomputation.
A key technical consideration of this workflow is the balance between compression and accuracy. Using fewer POD modes reduces the amount of data retained, but the reconstructed states must still provide sufficiently accurate sensitivities for the optimisation to produce a consistent result. CEEC therefore compared the approach with conventional checkpointing, examining both runtime and optimisation outcomes.

Results
The team evaluated the POD method first in a single-iteration study and then in complete optimisation runs. In the single-iteration study on one LUMI-G node, all tested POD configurations took less than 58% of the wall-clock time required by checkpointing. A configuration retaining 16 POD modes reproduced the design sensitivities closely and reduced the time associated with the relevant data-management operations by approximately 45%. The team then compared checkpointing and POD in two complete 100-iteration optimisations. With 32 POD modes, the results were:
| Optimisation case | Checkpointing | POD compression | Runtime reduction |
| Case 1 (maximise mixing) | 10 h 59 min | 7 h 51 min | ~29% |
| Case 2 (keep energy constant) | 11 h 03 min | 7 h 39 min | ~31% |
The final objective values from the POD-based runs were within 1.5% of their checkpointing reference results, and the convergence histories nearly coincided.
These results demonstrate that the method reduced runtime for the tested optimisation cases while preserving close agreement with the checkpointing reference. They do not establish that the same reduction will apply to every simulation or optimisation problem; performance and accuracy depend on the flow, the compression configuration, and the computational setup.


Impact on the Code
The implementation reduces the cost of managing time-dependent simulation data during adjoint-based topology optimisation. Rather than treating data storage and restoration as separate post-processing concerns, it integrates compression into the forward simulation and supplies reconstructed states to the adjoint calculation.
This approach can be particularly useful when simulations contain many time steps and the optimisation requires repeated forward and adjoint runs. In a separate production-scale configuration considered by the team, restoring forward states was estimated to represent a substantial part of the iteration cost. Removing much of this restoration work was estimated to reduce iteration cost by around 40%. This is a production-scale estimate, not the measured runtime reduction in the smaller optimisation cases reported above.
The POD method approach also makes use of CPU resources alongside GPU-based simulation, illustrating how heterogeneous computing resources can be combined to address a workflow bottleneck.
Contribution to the State of the Art
The technical advance is the integration of streaming POD compression into a GPU-based, time-dependent topology-optimisation workflow, with the compressed history used for adjoint sensitivity calculations.
CEEC advanced the data-management capability of adjoint-based topology optimisation by compressing forward flow history in situ, reducing runtime by approximately 29–31% in the tested complete optimisation runs while keeping final objective values within 1.5% of the checkpointing reference.
The results demonstrate a practical alternative to relying entirely on checkpointing and recomputation for the cases studied. Going forward, further evaluation would be needed to establish the best compression settings and accuracy-cost trade-offs for other flows and problem sizes.

Comments
One response to “In-situ Compression of Flow History Using Proper Orthogonal Decomposition for Faster Topology Optimisation”
[…] First, the team developed an in-situ flow-history compression approach based on POD. Instead of relying entirely on checkpointing and recomputing intermediate flow states during the adjoint simulation, the method compresses the flow history as the forward simulation runs. In two complete 100-iteration optimisation cases, this reduced runtime by approximately 29–31%, while keeping final objective values within 1.5% of the checkpointing reference results. Further details are presented in the success story “In-situ Compression of Flow History for Faster Topology Optimisation“. […]