Skip to content

OpenFOAM applications depending on libOpenFOAM.so fail to link since commit 73f6c7fe when compiling with g++ and WM_COMPILE_OPTION=Debug

Summary

When using the 'develop' branch from the OpenFOAM Git repository, since commit 73f6c7fe, all applications compiled with the g++ compiler using WM_COMPILE_OPTION=Debug cannot link with libOpenFOAM.so because of an undefined reference to `Foam::UPstream::selfComm'.

This problem is not present when using with the same compiler with the compilation optimization enabled (WM_COMPILE_OPTION=Opt)

Steps to reproduce

  • Checkout the latest version of the develop branch from the OpenFOAM Git repository : https://develop.openfoam.com/Development/openfoam/-/tree/develop?ref_type=heads
  • Define WM_COMPILE_OPTION=Debug in your file etc/prefs.sh
  • Recompile everything (libs and apps) from scratch
  • All link operations of applications (using /usr/bin/ld) that depend on libOpenFOAM.so will fail.
  • Using 'git bisect', one can isolate the problematic commit to 73f6c7fe

Example case

Error messages when trying to link the application laplacianFOAM with g++ in Debug mode:

`g++ -std=c++11 -m64 -pthread -DOPENFOAM=2302 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof -Wno-attributes -Wno-unknown-pragmas -O0 -fdefault-inline -ggdb3 -DFULLDEBUG -DNoRepository -ftemplate-depth-100 -I/scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/src/finiteVolume/lnInclude -I/scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/src/meshTools/lnInclude -iquote. -IlnInclude -I/scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/src/OpenFOAM/lnInclude -I/scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/src/OSspecific/POSIX/lnInclude -fPIC -Xlinker --add-needed -Xlinker --no-as-needed /scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/build/linux64GccDPInt32Debug/applications/solvers/basic/laplacianFoam/laplacianFoam.o -L/scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/platforms/linux64GccDPInt32Debug/lib
-lfiniteVolume -lfvOptions -lmeshTools -lOpenFOAM -ldl
-ggdb3 -DFULLDEBUG -lm -o /scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/platforms/linux64GccDPInt32Debug/bin/laplacianFoam

/usr/bin/ld: /scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/platforms/linux64GccDPInt32Debug/lib/libOpenFOAM.so: undefined reference to `Foam::UPstream::selfComm' collect2: error: ld returned 1 exit status make: *** [/scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/wmake/makefiles/general:170: /scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/platforms/linux64GccDPInt32Debug/bin/laplacianFoam] Error 1 `

What is the current bug behaviour?

Cannot link any applications depending on libOpenFOAM.so when compiling the 'develop' version of OpenFOAM in debug mode.

What is the expected correct behavior?

Proper linking of applications, as it is the case when the g++ compilation optimization is enabled (WM_COMPILE_OPTION=Opt)

Relevant logs and/or images

/usr/bin/ld: /scratch/cfd/openfoam/ESI-OpenFOAM-dev_bug/platforms/linux64GccDPInt32Debug/lib/libOpenFOAM.so: undefined reference to `Foam::UPstream::selfComm'

Environment information

  • OpenFOAM version : develop version using -DOPENFOAM=2302
  • Operating system : Fedora release 33
  • Hardware info : AMD Ryzen Threadripper 3970X
  • Compiler : g++ (GCC) versions 10.3.1, 12.3.0 and 13.1.0. All 3 versions of the g++ compilers have the same behaviour.

Possible fixes

The source of the problem is the following declaration of the static class member Foam::UPstream::selfComm in src/OpenFOAM/db/IOstreams/Pstreams/UPstream.H

      //- A communicator within the current rank only
      static constexpr label selfComm = 1;

This kind of declaration of a class static data member is allowed since C++11, but it looks like the proper implementation by GNU g++ is only done when the compilation optimization is enabled (-O1, -O2, -O3), probably by converting 'selfComm' to an inline declaration. This inline conversion is probably not done when the g++ optimization is disabled (-O0).

When invoking g++ with C++17 enabled, then the linking problem in Debug mode goes away, probably because the data member Foam::UPstream::selfComm gets inlined with or without the compiler code optimization being enabled.

See here : https://en.cppreference.com/w/cpp/language/static

And here: https://stackoverflow.com/questions/8016780/undefined-reference-to-static-constexpr-char

Possible solutions:

  • Use the C++17 flavour of the language with g++ (-std=c++17) instead of the C++11 variant (-std=c++11).
  • Declare and initialize the static data member the old way. For example, the following modifications will fix the compilation/linking problem:
diff --git a/src/OpenFOAM/db/IOstreams/Pstreams/UPstream.C b/src/OpenFOAM/db/IOstreams/Pstreams/UPstream.C
index e78777626f..9545ceb18b 100644
--- a/src/OpenFOAM/db/IOstreams/Pstreams/UPstream.C
+++ b/src/OpenFOAM/db/IOstreams/Pstreams/UPstream.C
@@ -61,6 +61,8 @@ Foam::UPstream::commsTypeNames
 namespace Foam
 {
 
+Foam::label Foam::UPstream::selfComm=1;
+
 // Determine host grouping.
 // Uses SHA1 of hostname instead of MPI_Comm_split or MPI_Comm_split_type
 // for two reasons:
diff --git a/src/OpenFOAM/db/IOstreams/Pstreams/UPstream.H b/src/OpenFOAM/db/IOstreams/Pstreams/UPstream.H
index acbd44c17d..f006e73f4d 100644
--- a/src/OpenFOAM/db/IOstreams/Pstreams/UPstream.H
+++ b/src/OpenFOAM/db/IOstreams/Pstreams/UPstream.H
@@ -318,7 +318,7 @@ public:
         static constexpr label globalComm = 0;
 
         //- A communicator within the current rank only
-        static constexpr label selfComm = 1;
+        static label selfComm;
 
         //- Communicator for all ranks, irrespective of any local worlds
         static constexpr label commGlobal() noexcept { return 0; }