Skip to content

redistributePar slow with preservePatches cosntraint

Follow-up to #2924, but not necessarily related in terms of what causes the issue.

To reliably decompose cyclic boundaries with redistributePar we currently need to use the preservePatches constraint for the cyclic boundaries. Unfortunately, this seems to cause a strong dip in performance of the utility. According to @Mattijs this should not really happen. I have first observed this on my own case, but now got some numbers using the periodicHill tutorial.

I use the steadyState case in the tutorial, bump the number of processors in decomposeParDict to 32, and accordingly switch to 4 partitions along y. Additionally, we add

constraints
{
    patches
    {
        //- Keep owner and neighbour on same processor for faces in patches
        //  (only makes sense for cyclic patches and cyclicAMI)
        type    preservePatches;
        patches (inlet outlet);
    }
}

which we can comment and uncomment to compare. Furthermore, it is better to make the case a bit larger, so in blockMeshDict we set the size of the hex to (200 160 400), and then run blockMesh.

Now, we decompose the case with

time mpirun -n 32 redistributePar -decompose -latestTime -parallel -fileHandler collated

On my machine, I get a 3x slowdown when preserving the patches.