redistributePar slow with preservePatches cosntraint
Follow-up to #2924, but not necessarily related in terms of what causes the issue.
To reliably decompose cyclic boundaries with redistributePar
we currently need to use the preservePatches
constraint for the cyclic boundaries. Unfortunately, this seems to cause a strong dip in performance of the utility. According to @Mattijs this should not really happen. I have first observed this on my own case, but now got some numbers using the periodicHill
tutorial.
I use the steadyState
case in the tutorial, bump the number of processors in decomposeParDict
to 32, and accordingly switch to 4 partitions along y. Additionally, we add
constraints
{
patches
{
//- Keep owner and neighbour on same processor for faces in patches
// (only makes sense for cyclic patches and cyclicAMI)
type preservePatches;
patches (inlet outlet);
}
}
which we can comment and uncomment to compare.
Furthermore, it is better to make the case a bit larger, so in blockMeshDict
we set the size of the hex
to (200 160 400)
, and then run blockMesh
.
Now, we decompose the case with
time mpirun -n 32 redistributePar -decompose -latestTime -parallel -fileHandler collated
On my machine, I get a 3x slowdown when preserving the patches.