CentOS precompiled packages don't include scotch
Hi,
I installed OpenFOAM v2006 with the instructions provided for CentOS in [1]. However, the decomposePar command is not working and gives the following error:
--> FOAM FATAL ERROR:
Attempted to use <scotch> without the scotchDecomp library loaded.
This message is from the dummy scotchDecomp stub library instead.
Please install <scotch> and ensure libscotch.so is in LD_LIBRARY_PATH.
The scotchDecomp library can then be built from src/parallel/decompose/scotchDecomp.
Dynamically loading or linking this library will add <scotch> as a decomposition method.
Scotch is installed in the system, but I suppose that there is something else missing. Does this has to do with the "Third-Party" builds (not included in the packages?).
[1] https://develop.openfoam.com/Development/openfoam/-/wikis/precompiled/redhat
Activity
Could be that the mpi libraries aren't loaded. @Pawan
@ctapia :
module load mpi/openmpi-x86_64
Try to run mpirun and see whether it is found by system .Edited by Pawan Ghildiyal@Pawan : I did that and it seems to be installed correctly:
$ mpirun -------------------------------------------------------------------------- mpirun could not find anything to do. It is possible that you forgot to specify how many processes to run via the "-np" argument.However, the error is still present.
mentioned in commit a777384c
-
@ctapia - thanks for bearing with us on resolving this. I think that I have most issues covered and will start a new build now. If all goes well, should be done in 3-6 hours, but I will need to check it again.
@mark : thanks for taking the time to looking into this. I have tested the newest build and it all seems to be working as expected :)
mentioned in commit 5e660c36
-
It is getting much closer, but the libptscotchDecomp still isn't getting packaged on centos8.
Edited by Mark OLESEN closed via commit 5e660c36
@mark Hello. I'm having this issue again on pre-compiled v2106. The system is Rocky Linux 8.4(should be identical to CentOS 8).
Openmpi is enabled. Scotch and Metis are installed, but they're not recognized.
# mpirun --version mpirun (Open MPI) 4.0.5 Report bugs to http://www.open-mpi.org/community/help/ # dnf install scotch scotch-devel metis metis-devel Last metadata expiration check: 4:14:27 ago on Mon 26 Jul 2021 06:46:37 AM EDT. Package scotch-6.0.5-3.el8.x86_64 is already installed. Package scotch-devel-6.0.5-3.el8.x86_64 is already installed. Package metis-5.1.0-17.el8.x86_64 is already installed. Package metis-devel-5.1.0-17.el8.x86_64 is already installed. Dependencies resolved. Nothing to do. Complete!The exact same error message:
--> FOAM FATAL ERROR: (openfoam-2106) Attempted to use <scotch> without the scotchDecomp library loaded. This message is from the dummy scotchDecomp stub library instead. Please install <scotch> and ensure libscotch.so is in LD_LIBRARY_PATH. The scotchDecomp library can then be built from src/parallel/decompose/scotchDecomp. Dynamically loading or linking this library will add <scotch> as a decomposition method. From virtual Foam::labelList Foam::scotchDecomp::decompose(const Foam::polyMesh&, const pointField&, const scalarField&) const in file dummyScotchDecomp.C at line 102. FOAM exitingEdited by Guanyang Xue-
Hi @xuegy
Nice to know that rockylinux is now out in production. Not quite so good to find that the scotch dependency has somehow slipped through again. Will take a look.
I noticed that the log file for RHEL 7 shows
scotchDecompis properly compiled.# Detection for 'scotch' scotch (label=int) - /usr ptscotch - /usr scotch=true root="/usr" include="/usr/include" library="/usr/lib64" ptscotch=true root="/usr" include="/usr/include/openmpi-x86_64" library="/usr/lib64/openmpi/lib" scotch (label=int) - /usr wmake scotchDecomp dep: scotchDecomp.C Ctoo: scotchDecomp.C ld: /builddir/build/BUILD/OpenFOAM-v2106/platforms/linux64GccDPInt32Opt/lib/libscotchDecomp.so scotch (label=int) - /usr ptscotch - /usr wmake ptscotchDecomp (mpi=SYSTEMOPENMPI:sys-openmpi) wmake ptscotchDecomp dep: ptscotchDecomp.C Ctoo: ptscotchDecomp.C ld: /builddir/build/BUILD/OpenFOAM-v2106/platforms/linux64GccDPInt32Opt/lib/sys-openmpi/libptscotchDecomp.soWhile for RHEL 8 there's something wrong with the header.
# Detection for 'scotch' ==> skip scotch (label='32', scotch.h has 'unknown') scotch=false root="" include="" library="" ptscotch=false root="" include="" library="" ==> skip scotch (label='32', scotch.h has 'unknown') ==> skip scotch (label='32', scotch.h has 'unknown')By the way, after fixing scotch, shall we have metis enabled as well?
Edited by Guanyang Xue-
It looks like RedHat does their own thing:
#if __WORDSIZE == 32 #include "scotch-32.h" #elif __WORDSIZE == 64 #include "scotch-64.h" #else #error "Unknown word size" #endifFrom the include/bits/wordsize.h, will mostly define
__WORDSIZE 64(for x86-64), which then apparently includes scotch-64.h vs scotch-32.hLooks like we will need extra detection for this and also need to rework the decomposition interface to automatically handle the translation from label-size = 32 to int64 !!
Edited by Mark OLESEN So the scotch via package manager uses int64 by default? If OpenFOAM is compiled in 32bit label will they be compatible?
Edited by Guanyang Xue-
Hi @xuegy That was a misread from my part. The 32/64 difference turns out to be LP64 vs ILP32 so in both cases will have int32 for addressing the SCOTCH_Num (ie, no worries).
mentioned in commit 395b680c
