B10 occDrivAerStaticMesh: fixedIter doesn't fix coarsest level of the multi-grid solver
During testing of occDrivAerStaticMesh we found inconsistent behavior between runs with fixed tolerances and iteration numbers. This seems to be caused by the coarsest level correction of the multi-grid solver. When running with the GAMG debug switch, e.g. srun -n 512 simpleFoam -parallel -debug-switch GAMG=1, fixedTol log is OK:
GAMGSolver settings : cacheAgglomeration:1 nPreSweeps:0 preSweepsLevelMultiplier:1 maxPreSweeps:4 nPostSweeps:2 postSweepsLevelMultiplier:1
maxPostSweeps:4 nFinestSweeps:2 interpolateCorrection:0 scaleCorrection:1 directSolveCoarsest:0
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 0.008027239019849546, No Iterations 25
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 0.009713740903548349, No Iterations 31
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 0.009535146848767053, No Iterations 34
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 0.009153981339444102, No Iterations 47
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 0.009830741343943128, No Iterations 38
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 0.009441290217225807, No Iterations 47
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 0.008521596653701715, No Iterations 40
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 0.009860497527849961, No Iterations 47
GAMG: Solving for p, Initial residual = 0.003275135124737028, Final residual = 3.274223140068516e-05, No Iterations 8
but fixedIter always ends up with 1000 iterations:
GAMGSolver settings : cacheAgglomeration:1 nPreSweeps:0 preSweepsLevelMultiplier:1 maxPreSweeps:4 nPostSweeps:2 postSweepsLevelMultiplier:1
maxPostSweeps:4 nFinestSweeps:2 interpolateCorrection:0 scaleCorrection:1 directSolveCoarsest:0
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 1.079794721644342e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 1.920687426374025e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 3.405838458576473e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 2.309646584342756e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 9.219459896262679e-81, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 4.009746068126394e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 1.803592681739625e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 2.08758244943197e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 1.816064787355942e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 3.48877995253826e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 1.827946834753762e-80, No Iterations 1000
DICPCG: Solving for coarsestLevelCorr, Initial residual = 1, Final residual = 4.224220482428126e-80, No Iterations 1000
GAMG: Solving for p, Initial residual = 0.003083580301485853, Final residual = 1.015363616717244e-05, No Iterations 12
Probably, coarsestLevelCorr picks up tolerance settings from the parent but not the maxIter value.
The setup needs something similar to what we did for the conical diffuser in exaFOAM. It's a bit different since GAMG there is used as a preconditioner but the fix is something like
p
{
solver GAMG;
smoother GaussSeidel;
tolerance 0;
relTol 0;
maxIter 12;
coarsestLevelCorr
{
solver PCG;
preconditioner DIC;
minIter ??;
maxIter ??;
}
}
We need only to decide what to put instead of "??".
@mark.wasserman, can you please run the fixedTol case, which the fixedIter is derived from, with the debug switch from above and provide a mean value for the coarsest level iteration number?