Modules issueshttps://develop.openfoam.com/groups/modules/-/issues2022-04-29T16:19:07Zhttps://develop.openfoam.com/modules/external-solver/-/issues/22improve readme with information on how to setup the environment before runnin...2022-04-29T16:19:07Zstefano zampiniimprove readme with information on how to setup the environment before running AllwmakeWhen I first compiled OpenFOAM, I edited the config file for PETSc and have FOAM find the library.
However, such information or any other possible way to customize the environment before compiling petsc4FOAM is missing in the README file...When I first compiled OpenFOAM, I edited the config file for PETSc and have FOAM find the library.
However, such information or any other possible way to customize the environment before compiling petsc4FOAM is missing in the README file. Alternatives to set up the environment before calling Allwmake would be useful for new users. For example, is it possible to switch PETSc installation (say, from debug to optimized build) and just run Allwmake again? If yes, what are the steps?Mark OLESENMark OLESENhttps://develop.openfoam.com/modules/visualization/-/issues/6Symbols not found at runtime2021-07-23T13:37:44ZSimone BnaSymbols not found at runtime@mark OpenFOAM crashes when it loads the visualization libraries (catalystFoam and runTimePostProcessing) since it does not find some python, vtk and paraview symbols.
Solved by adding explicitly the dependency on the python libraries a...@mark OpenFOAM crashes when it loads the visualization libraries (catalystFoam and runTimePostProcessing) since it does not find some python, vtk and paraview symbols.
Solved by adding explicitly the dependency on the python libraries and fixing the list of vtk and paraview libraries.https://develop.openfoam.com/modules/external-solver/-/issues/18Question about petsc4Foam performance2022-04-13T08:35:55Zstefano zampiniQuestion about petsc4Foam performanceDear zampini,
Hi:) I am a UCL student who is working on the performance optimization of openFoam based on petsc. When I tried to use petsc4Foam to replace the original openFoam solvers in cavity case, I found that the solving speed of p...Dear zampini,
Hi:) I am a UCL student who is working on the performance optimization of openFoam based on petsc. When I tried to use petsc4Foam to replace the original openFoam solvers in cavity case, I found that the solving speed of petsc solver was much slower than the original one. However, in your essay: A library to plug in PETSC into the OpenFOAM framework, the petsc method should be better than the openFoam one. So I am wondering did I set any parameters wrong? And could you give me any idea about the performance of petsc4foam? I thought petsc should run faster than the original solver even in 1 node. Thanks a lot for your time!
BlockMesh: 100x100x100
ControlDict: deltaT: 0.001, endTime:0.5
Tolerance for P: 1e-04
relTol : 0
maxIter: 3000
Parameters for petsc+foam:
P: bjacobi +icc+cg
U: DILU+PBiCGStab
Parameters for original openFoam:
P: FDIC+pcg
U: symGaussSeidel+smoothSolver
Environment:
1 node: AMG 7742 with 128 cores.
GCC: 9.2.0
MPI: OpenMPI 4.0.3
Thanks,
Xiaofeng Fustefano zampinistefano zampinihttps://develop.openfoam.com/modules/external-solver/-/issues/40PETSc4FOAM with OpenFOAM.org2023-10-09T14:38:42ZFerran de MiguelPETSc4FOAM with OpenFOAM.orgHi, is there a way of using this library and PETSc with the OpenFOAM.org version (i.e. OF10) instead of the OpenFOAM.com (i.e. OF2306)?
I've investigated and the Allwmake checks for some scripts that are not present in the OpenFOAM.org v...Hi, is there a way of using this library and PETSc with the OpenFOAM.org version (i.e. OF10) instead of the OpenFOAM.com (i.e. OF2306)?
I've investigated and the Allwmake checks for some scripts that are not present in the OpenFOAM.org version such as the "have_petsc". Would this be solved by simply copying all the required files and scripts into an OpenFOAM 10 installation? Thanks for your help.https://develop.openfoam.com/modules/external-solver/-/issues/38solve a special matrix with hypre2023-06-21T00:23:04ZYu Hesolve a special matrix with hypreI have trouble using hypre to solve a linear equation with a special shape of matrix, so I come here looking for help.
The matrix has the shape like
[A -B^T;
B O;
O C]
where A is dense in general. B^T is transpose of B. B is pretty spar...I have trouble using hypre to solve a linear equation with a special shape of matrix, so I come here looking for help.
The matrix has the shape like
[A -B^T;
B O;
O C]
where A is dense in general. B^T is transpose of B. B is pretty sparse and has only very few 1 or -1 in each of its column. C is another very sparse matrix which has one or two 1 or -1 in each of its row. The following is a small example:
[1e-4 3e-5 0 0 -1 1;
3e-5 1e-4 -1 1 0 0;
0 1 0 0 0 0;
0 -1 0 0 0 0;
1 0 0 0 0 0;
-1 0 0 0 0 0;
0 0 0 -1 1 0;
0 0 0 1 0 0]
The rhs is a vector with very few 1 or -1. Like
[0 0 1 -1 0 0 0 0]
I have tried various options to solve this linear problem but cannot find a realiable solution. ILU works for the above example, but is not reliable. When I turn to another case it fails. I am not an expert user of hypre so I would like to see if there are any recommendations for the best setup to solve this kind of problem. Thanks for your help.https://develop.openfoam.com/modules/external-solver/-/issues/37Problem with finding petscFoam after compiling OF v2012 with PETSc developmen...2023-05-17T09:35:37Ztay wee bengProblem with finding petscFoam after compiling OF v2012 with PETSc development verHi, I tried to compile with the above software and version. Managed to get the lib:
```
[tsl@as-login-01 external-solver]$ ls ~/OpenFOAM/tsltaywb-v2012/platforms/linux64Gcc112DPInt32Opt/lib/
libpetscFoam.so
```
However there's an err...Hi, I tried to compile with the above software and version. Managed to get the lib:
```
[tsl@as-login-01 external-solver]$ ls ~/OpenFOAM/tsltaywb-v2012/platforms/linux64Gcc112DPInt32Opt/lib/
libpetscFoam.so
```
However there's an error saying:
`/usr/bin/ld: warning: libcuda.so.1, needed by /home/project/11001487/source/petsc_dev/petsc-dev_gnu_gpu_debug/lib/libpetsc.so, not found (try using -rpath or -rpath-link)`
I tried:
`foamHasLibrary -verbose petscFoam`
and
`eval $(foamEtcFile -sh -config petsc -- -force)`
but it can't be found:
```
Could not load "petscFoam"
libpetsc.so.3.019: cannot open shared object file: No such file or directory
```
May I know why?
Thanks.https://develop.openfoam.com/modules/external-solver/-/issues/36Running PETsc4FOAM with multiple GPUs2023-04-06T11:28:54ZElbert HoRunning PETsc4FOAM with multiple GPUsHello, I am a SDSU student who is working on comparing results between using PETsc with multiple GPUs and running the solver by itself in parallel with CPUs. I am able to run my solver on one GPU, but I am confused as to how to run it on...Hello, I am a SDSU student who is working on comparing results between using PETsc with multiple GPUs and running the solver by itself in parallel with CPUs. I am able to run my solver on one GPU, but I am confused as to how to run it on multiple. Is there a solution, or a user guide for this? I am running OpenFOAM v2006, and my solver is a variant of icoreactingmultiphaseinterfoam.
Best Regards,
Elbert Hohttps://develop.openfoam.com/modules/external-solver/-/issues/35running ./Allwmake of external-solver inside openfoam v19122022-12-19T13:04:47ZDisinlung Kameirunning ./Allwmake of external-solver inside openfoam v1912Hi all i have build Petsc
and have set the path as shown below
```
export PETSC_DIR=/home/urban/projects/petsc
export PETSC_ARCH=arch-linux-c-debug
export LD_LIBRARY_PATH=$PETSC_DIR/$PETSC_ARCH/lib:$LD_LIBRARY_PATH
export PATH=$PETSC_D...Hi all i have build Petsc
and have set the path as shown below
```
export PETSC_DIR=/home/urban/projects/petsc
export PETSC_ARCH=arch-linux-c-debug
export LD_LIBRARY_PATH=$PETSC_DIR/$PETSC_ARCH/lib:$LD_LIBRARY_PATH
export PATH=$PETSC_DIR/$PETSC_ARCH/bin:$PATH
export PETSC_ARCH_PATH=$PETSC_DIR/$PETSC_ARCH
export INCLUDE=$PETSC_DIR/$PETSC_ARCH/include:$INCLUDE
```
then i
`source Openfoam-v1912/etc/bashrc`
then
i go to
`Openfoam-v1912/modules/external-solver/`
and i do
`./Allwmake `
and i got the result as shown below
```
[urban@login02 external-solver]$ ./Allwmake
========================================
2022-12-12 17:45:15 +0530
Starting compile of external-solver with OpenFOAM-v1912_try
Gcc system compiler
linux64GccDPInt32Opt, with SYSTEMOPENMPI openmpi-system
prefix = default (user)
==> skip petsc (no header)
2022-12-12 17:45:15 +0530
========================================
Finished compile of external-solver with OpenFOAM-v1912_try
Gcc system compiler
linux64GccDPInt32Opt, with SYSTEMOPENMPI openmpi-system
```
now openfoam fail to find the command
`petscFoam`
can someone kindly tell me whether my petsc4Foam is installed or not
Thankshttps://develop.openfoam.com/modules/external-solver/-/issues/33petsc4foam & foam2csr - Issue with AmgXCSRMatrix::setValuesLDU2022-11-25T03:47:52ZVansh Sharmapetsc4foam & foam2csr - Issue with AmgXCSRMatrix::setValuesLDUHi!
I am trying to install petsc4Foam using the amgxwrapper branch [git clone --branch amgxwrapper https://develop.openfoam.com/modules/external-solver.git petsc4foam]. I have installed AMGX-2.2.0, OpenFOAM v2112, PETSC-v3.15.5 and foam...Hi!
I am trying to install petsc4Foam using the amgxwrapper branch [git clone --branch amgxwrapper https://develop.openfoam.com/modules/external-solver.git petsc4foam]. I have installed AMGX-2.2.0, OpenFOAM v2112, PETSC-v3.15.5 and foam2csr. (I tried with PETSc-v3.16.6 and v3.16.2)
I changed the solvers/petscSolver.C and solvers/petscSolver.H (downloaded from here) after facing the initialization error for List<label>& lowNonZero = ctx.lowNonZero;
When I run ./Allmake command in the petsc4Foam folder, I encounter the following error.
I could not find any changes made to the amgxSolver.C file post commit #29 to the petsc4foam main branch.
I've tried changing the variable types in the solvers/amgxSolver.C [lines 269-404] but I am not sure if that's the right approach for this.
Would really appreciate some guidance from the developers. @sbna @szampini @mmartineau
```
solvers/amgxSolver.C:404:5: error: no matching function for call to ‘AmgXCSRMatrix::setValuesLDU(const label&, const label&, Foam::label&, Foam::label&, Foam::label&, const long int*, const long int*, Foam::label&, long int*, long int*, const double*, const double*, const double*, double*)’
);
^
In file included from solvers/amgxSolver.H:46:0,
from solvers/amgxSolver.C:38:
foam2csr/src/AmgXCSRMatrix.H:54:14: note: candidate: void AmgXCSRMatrix::setValuesLDU(int, int, int, int, int, const int*, const int*, int, const int*, const int*, const float*, const float*, const float*, const float*)
void setValuesLDU
^~~~~~~~~~~~
foam2csr/src/AmgXCSRMatrix.H:54:14: note: no known conversion for argument 6 from ‘const long int*’ to ‘const int*’
foam2csr/src/AmgXCSRMatrix.H:74:14: note: candidate: void AmgXCSRMatrix::setValuesLDU(int, int, int, int, int, const int*, const int*, int, const int*, const int*, const double*, const double*, const double*, const double*)
void setValuesLDU
^~~~~~~~~~~~
foam2csr/src/AmgXCSRMatrix.H:74:14: note: no known conversion for argument 6 from ‘const long int*’ to ‘const int*’
```https://develop.openfoam.com/modules/external-solver/-/issues/28Config for building PETSc with MPI and multiple MPI versions2021-10-19T03:13:09ZDavide FransosConfig for building PETSc with MPI and multiple MPI versionsI am building PETSc with `--with-mpi-dir=MY_MPI_INSTALL` following the instructions in the README.
The configuration is not complete though, as building `external-solver` with `Allwmake` does not include the MPI headers required by PET...I am building PETSc with `--with-mpi-dir=MY_MPI_INSTALL` following the instructions in the README.
The configuration is not complete though, as building `external-solver` with `Allwmake` does not include the MPI headers required by PETSc headers and I have to manually add an include entry for my MPI installation in petsc4foam/Make/options. With that, everything builds correctly, but it's not what I expect.
Also, the built lib goes into `FOAM_MODULE_LIBBIN/$WM_OPTIONS` and not into `FOAM_MODULE_LIBBIN/$WM_OPTIONS/$FOAM_MPI`. This prevents from using multiple MPI versions with the same OpenFOAM build, unless we play with `FOAM_MODULE_LIBBIN` to get there.
Questions:
1. am I correct in building PETSc with a specific MPI or does petsc4foam rely on PStream?
2. in case I'm doing right, how should I better handle the multiple MPI configuration?https://develop.openfoam.com/modules/external-solver/-/issues/27Unknown symmertric solver "petsc"- Clariffy steps for correctly building and ...2021-09-20T15:49:38ZFelix LeClairUnknown symmertric solver "petsc"- Clariffy steps for correctly building and linking PETScCurrently having issues getting up and running with petsc4foam
edit: this portion of the issue was due to the script looking for a hard link for petsc 3.15.0, where as I had 3.15.4.
New/subsequent issue is that solver "petsc" is not ...Currently having issues getting up and running with petsc4foam
edit: this portion of the issue was due to the script looking for a hard link for petsc 3.15.0, where as I had 3.15.4.
New/subsequent issue is that solver "petsc" is not found. See reply 2
Specifically I don't seem to be correctly getting `libpetscFoam.so`.
configuration for petsc is:
` ./configure --with-64-bit-indices=0 --with-precision=double --with-debugging=0 --COPTFLAGS=-O3 --CXXOPTFLAGS=-O3 --FOPTFLAGS=-O3 --prefix=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER$WM_PRECISION_OPTION$WM_LABEL_OPTION/petsc-git PETSC_ARCH=$WM_OPTIONS --download-openblas --download-mpich --with-fc=gfortran --prefix=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER$WM_PRECISION_OPTION$WM_LABEL_OPTION/petsc-git PETSC_ARCH=$WM_OPTIONS --download-scotch --hip=1 `
after successfully building openfoam2106 from source
I've attempted to run the ./Allwmake script from this repository, which returned
```
/Allwmake
========================================
2021-09-14 18:04:38 -0400
Starting compile of external-solver (petsc) with OpenFOAM-v2106
Gcc system compiler
linux64GccDPInt32Opt, with SYSTEMOPENMPI sys-openmpi
prefix = default (user)
==> skip petsc (no header)
2021-09-14 18:04:38 -0400
========================================
Finished compile of external-solver (petsc) with OpenFOAM-v2106
Gcc system compiler
linux64GccDPInt32Opt, with SYSTEMOPENMPI sys-openmpi
```
The external solver script does seem to detect petsc correctly, but the library itself is missing.
This leads me to suspect that the issue lies with the configuration in petsc not compiling the needed lib.
Running f`ind . -name "*oam.so"` from the top of the petsc source directory and openfoam source directory returned no relevant hits.
Any advice is greatly appreciatedhttps://develop.openfoam.com/modules/visualization/-/issues/8Prototype an update to Catalyst V2 APIs2021-08-10T09:16:56ZRobert Manson-SawkoPrototype an update to Catalyst V2 APIs@mark,
This is a carry over from [ParaView discourse](https://discourse.paraview.org/t/catalyst-v2-for-openfoam/6903) exchange we had a few months ago. I wasn't able to technically work on the stub I created back then, but I would like ...@mark,
This is a carry over from [ParaView discourse](https://discourse.paraview.org/t/catalyst-v2-for-openfoam/6903) exchange we had a few months ago. I wasn't able to technically work on the stub I created back then, but I would like to now return to this project and make some progress this calendar year. Moreover, we are working to secure backing from our current research programme for a 9month demo project which will cover half of my time and another colleague from UKRI. The objective is to adapt work-flows in both organisations to use in-situ and evaluate their performance on both cloud and on-prem instances.
This message is just to gauge your interest. At this stage, I am only asking whether you would like to see any scaling or performance analysis results we may come up with.
Also, I have read your [blog post](https://olesenm.github.io/2020/11/11/openfoam-code-provenance/) and in the long run we would like to be able to contribute to visualisation module (if we develop anything worth contributing of course).Mark OLESENMark OLESENhttps://develop.openfoam.com/modules/external-solver/-/issues/26petsc4Foam : handle cyclics implicitly2021-07-13T21:18:37ZMattijs Janssens4-Mattijs@users.noreply.develop.openfoam.competsc4Foam : handle cyclics implicitlyIs it possible to handle cyclic implicitly? I'd like to solve the velocity component-coupled ('coupled yes' in fvSolution) and have cyclics with a rotation. Can petsc4Foam do this?
@szampiniIs it possible to handle cyclic implicitly? I'd like to solve the velocity component-coupled ('coupled yes' in fvSolution) and have cyclics with a rotation. Can petsc4Foam do this?
@szampinihttps://develop.openfoam.com/modules/external-solver/-/issues/25Missing header file for amgxwrapper2023-12-15T03:25:40ZWinston VirtausMissing header file for amgxwrapperHi!
I’ve been successfully running GPU accelerated solvers via petsc4foam for three months now and decided to give the amgxwrapper branch of this project a try to see if there were any performance gains over PETSc.
Amgxwrapper compilat...Hi!
I’ve been successfully running GPU accelerated solvers via petsc4foam for three months now and decided to give the amgxwrapper branch of this project a try to see if there were any performance gains over PETSc.
Amgxwrapper compilation starts fine on OF2012 but at some point the compiler notifies that AmgXCSRMatrix.H file is missing. However, I cannot seem to locate this file anywhere.
Has this AmgXCSRMatrix.H header file been made publicly available?https://develop.openfoam.com/modules/visualization/-/issues/4Installation issue with CUDA 10 + Gcc82021-01-15T15:49:32ZSimone BnaInstallation issue with CUDA 10 + Gcc8@mark As reported in this issue, https://github.com/LLNL/blt/issues/341, check for working CUDA compiler fails (Cuda10 + Gcc8).
I solved on Marconi100 by adding -DCMAKE_CUDA_FLAGS="-std=c++11" to cmakeOpts
cmakeOpts="-DCMAKE_INSTALL...@mark As reported in this issue, https://github.com/LLNL/blt/issues/341, check for working CUDA compiler fails (Cuda10 + Gcc8).
I solved on Marconi100 by adding -DCMAKE_CUDA_FLAGS="-std=c++11" to cmakeOpts
cmakeOpts="-DCMAKE_INSTALL_PREFIX=$CMAKE_INSTALL_PREFIX -DCMAKE_CUDA_FLAGS="-std=c++11""https://develop.openfoam.com/modules/visualization/-/issues/3integration of temporal caching2021-01-15T15:45:21ZMark OLESENintegration of temporal cachingMentioned in email chain with @sbna, @IvanSpisso David Demarle and @acbauer etc.
From Dave
> My changes to update openFOAM for Catalyst v5.9 (at least the recently legacy mode) and adding in the new temporal option are available here: h...Mentioned in email chain with @sbna, @IvanSpisso David Demarle and @acbauer etc.
From Dave
> My changes to update openFOAM for Catalyst v5.9 (at least the recently legacy mode) and adding in the new temporal option are available here: https://github.com/demarle/openfoamvismodule. Also etc/casedicts/insitu/catalyst/printChannels.py needs python3 print()s.
> As described in the video I just used the propeller tutorial with a fancied up Catalyst script to do OSPRay path traced rendering and the cache-rewind-revise-playback-and-continue trick to exercise all of that beautiful memory capacity. My changes to the case files are attached.
> If you attended SC you can see my recorded presentation here:
https://www.eventscribe.net/2020/includes/tracking/clickTracking.asp?lfp=R0M2WEp6UWtPdll4czk2T1hSVFlROGUvY2ZWWTFyWklyY2ZacVgrZUk0L2pBTWdPNm1GcmF5NVpGYlpLN2hzRXVZTlo1dVkyam13NGJ0bmVKOUc5dWVyYjEvdlIxdEFpZE02OVA5M2dKUGdVZko1eEQ1SE9semljMVBJVUFCNkNVSlN4VGZnSHlSVkFwNnR4YmZLVEl5YWVKS0NVcTN4YVR1OXU5bzdhWFRYdDlEamF0NWorbkt3WE95MlorTCtzOUp2SGpQWkFwc3FCajlJeTcyU25zZG5RR2NsWXVqdXd3bmtOUXBrTHZNQT0=
> Otherwise the slides are here:
https://drive.google.com/file/d/12ilOYNBfcDYgIaOOfW4eO_o1npAi7Eib/view?usp=sharinghttps://develop.openfoam.com/modules/visualization/-/issues/1runtTimePostProcessing issues with interpolated2020-06-24T13:48:50ZMark OLESENruntTimePostProcessing issues with interpolatedAs @Prashant noted during testing, the unlikely (undesirable) combination
- a sampled cutting plane with interpolate = true
- rendering in parallel
The result looks like rubbish.
If we switch to VTK rendering in serial - works fine, so...As @Prashant noted during testing, the unlikely (undesirable) combination
- a sampled cutting plane with interpolate = true
- rendering in parallel
The result looks like rubbish.
If we switch to VTK rendering in serial - works fine, so the problem is not in the sampler.
Normally we would use 'interpolate = false' when sampling, but use 'smooth = true' for viz to get pretty pictures.
Needs more investigationMark OLESENMark OLESEN