Incorrect total Amgx Iteration number and final residual return
In each AmgX solver in OpenFOAM, the total iterations and last residual values are returned incorrectly.
Time = 2
smoothSolver: Solving for Ux, Initial residual = 0.0881388, Final residual = 0.00728349, No Iterations 10
smoothSolver: Solving for Uy, Initial residual = 0.489208, Final residual = 0.0413399, No Iterations 9
smoothSolver: Solving for Uz, Initial residual = 0.29561, Final residual = 0.0239977, No Iterations 9
Offloaded LDU matrix values (only) on CUDA device and converted to CSR
iter Mem Usage (GB) residual rate
----------------------------------------------------------------------
Ini 0.64447 4.398599e+00
0 0.64447 1.931445e+00 0.4391
1 0.6445 1.088411e+00 0.5635
2 0.6445 6.510092e-01 0.5981
3 0.6445 4.320142e-01 0.6636
4 0.6445 2.844746e-01 0.6585
5 0.6445 1.763286e-01 0.6198
6 0.6445 1.331400e-01 0.7551
7 0.6445 1.035108e-01 0.7775
8 0.6445 6.456993e-02 0.6238
9 0.6445 4.532301e-02 0.7019
10 0.6445 2.859637e-02 0.6309
----------------------------------------------------------------------
Total Iterations: 11
Avg Convergence Rate: 0.6327
Final Residual: 2.859637e-02
Total Reduction in Residual: 6.501245e-03
Maximum Memory Usage: 0.644 GB
----------------------------------------------------------------------
Total Time: 0.0670263
setup: 0.0229755 s
solve: 0.0440508 s
solve(per iteration): 0.00400462 s
PETSc-AMGx: Solving for p, Initial residual = 4.3986, Final residual = 0.045323, No Iterations 10
time step continuity errors : sum local = 0.00462175, global = -0.000166349, cumulative = -0.000110222
smoothSolver: Solving for omega, Initial residual = 0.0138737, Final residual = 0.000939541, No Iterations 3As showin in the above PETSc-AMGx solver log, the Total Iterations: 11 in the AmgX solver, but the No Iterations 10 in the OpenFOAM log should show 11
PETSc-AMGx: Solving for p, Initial residual = 4.3986, Final residual = 0.045323, No Iterations 10The Final residual = 0.45323 is not the last residual in the AmgX solver but the 9th redisual value is shown.
9 0.6445 4.532301e-02 0.7019This inconsistency occurs from the index mismatch in the AmgX source.
If the source code changeed as ctx.performance.nIterations() = nIters and amgx.getResidual(nIters, rnorm)the OpenFOAM residual log shows correct result.
Offloaded LDU matrix values (only) on CUDA device and converted to CSR
iter Mem Usage (GB) residual rate
----------------------------------------------------------------------
Ini 0.712158 4.436648e+00
0 0.712158 1.930312e+00 0.4351
1 0.7122 1.069685e+00 0.5542
2 0.7122 6.444335e-01 0.6025
3 0.7122 4.259357e-01 0.6609
4 0.7122 2.859598e-01 0.6714
5 0.7122 1.781309e-01 0.6229
6 0.7122 1.334149e-01 0.7490
7 0.7122 1.023903e-01 0.7675
8 0.7122 6.363197e-02 0.6215
9 0.7122 4.391825e-02 0.6902
----------------------------------------------------------------------
Total Iterations: 10
Avg Convergence Rate: 0.6303
Final Residual: 4.391825e-02
Total Reduction in Residual: 9.898971e-03
Maximum Memory Usage: 0.712 GB
----------------------------------------------------------------------
Total Time: 0.0604765
setup: 0.0222095 s
solve: 0.038267 s
solve(per iteration): 0.0038267 s
PETSc-AMGx: Solving for p, Initial residual = 4.43665, Final residual = 0.0439182, No Iterations 10In the above OpenFOAM log, the Total Iteration: 10 and final residual 4.391825e-02 are correctly returned to the OpenFOAM log.