Release coherent mesh format improving parallel I/O performance

Functionality to add/problem to solve

Improve I/O performance and handling in parallel runs:

• Decrease time-to-solution (mainly due to faster pre-processing)
• No reconstruction for post-processing
• Edit a single ASCII file to alter a BC on all processors
• Reduce substantially number of files (i.e. inodes)

Target audience

All types of cases run in parallel.

Proposal

• Use the implementation of the coherent mesh format developed during the exaFOAM project that has already been integrated into OpenFOAM: https://gitlab.com/openfoam/community/exafoam/io/-/tree/coherent-preview.v2412 (note the rolling branches).
• Define missing features for the minimum viable product.
• Prepare and execute a roadmap to finally integrate the coherent mesh format into the release branch.

What does success look like, and how can we measure that?

Up to 15x faster serial pre-processing (decompose + renumber) of the DrivAer case from the 1st OpenFOAM HPC Challenge with 236 million cells when compared to the collated format. See this presentation: https://github.com/OpenFOAM-HPC-Challenge/OHC1/blob/main/presentations/01-Wikki-SWTrack-IO.pdf

31x lower Cost-To-Solution for the Combustor Grand Challenge with 489 million cells decomposed into 32,768 partitions of the pre- and post-processing compared to the collated format. See the documentation of the case: https://develop.openfoam.com/committees/hpc/-/tree/develop/combustion/XiFoam/DLRCJH#pre--and-post-processing

Links / references

Paper describing the format: https://www.researchsquare.com/article/rs-3897818/v1

Working implementation: https://gitlab.com/openfoam/community/exafoam/io/-/tree/coherent-preview.v2412