Case sensitive filesystems, Darwin builds

This will hit you just about everywhere. You really need to have a case-sensitive filesystem for OpenFOAM. Generally we have an UpperCase for templated classes and the lowerCase for the template instances.

Eg, PrimitivePatch as template, primitivePatch for a concrete type.

I don't know how many more there are like that, but quite a few.

Answer = case-sensitive filesystem. I thought that the newer OSX shipped with that?

Thanks a lot, Mark. I don't know if Catalina is case-sensitive, but anyway, don't worry, I just created a case sensitive DMG disk image (DMG images are a pretty powerful feature for development in MacOS) and extracted the tarball there. No problems now.

But then I realized that Darwin support was removed. I looked at the commits, but the issues mentioned at BuildIssues.md were not so serious (just -lrt in scotch/ptscotch, and gmp/mpfr in CGAL), two things that I believe I can overcome easily (in fact maybe gmp/mpfr are going to work for me without removing the wmake settings, because I compiled them manually -and all their tests passed-, but anyway if I face more problems I'll remove them).

Anyway, I removed v1912 and downloaded v1812 because I believe it's the last version with MacOS support (yes, it says it wasn't very tested, but if you were able to realize about -lrt and gmp/mpfr, then I guess you were able to build it, and you don't mention test failures, so I'm not seeing anything specially worrying).

Note that I already built all thirdparty dependencies on my own (except Paraview, but I'll try to point wmake to use a precompiled Paraview and the source code for its same version: I mean, if the headers are in the Paraview source code, and if the plugins are in a directory in the precompiled Paraview, I believe it should work, but if it doesn't I'll build it myself, it's not so hard, being from the creators of CMake :-)

Kind regards, and thanks a lot!

Oops, my fault, nothing was removed: I'll try with v1912 and I'll report back. I'll do it tomorrow. I'm assuming the build is in-source (not out-of-source), so I'll need a case-sensitive mount: I'll create for example a 16GB ramdisk in a Mac with 32GB RAM tomorrow (formatted as case-sensitive, I believe 16GB will be enough, as the source is only about 400MB and I don't think binaries are going to be that huge).

Anyway, if you wish to tell me any advice before I start tomorrow, please do!

assigned to @mark

If you find anything "non-obvious" in your build process, let me know here and I can add it into the

build wiki OSX section

Build support removed for 1912? This sounds like a mistake somewhere. In documentation, files or something. AFAIK there should be no difference with 1912. Maybe just that it was added in 1812 and thus made a bigger fuss of it and then no mention since then since it simply is there?

In commit 09cf9d99 - just dropped the special compiler-define, since we can figure it out other ways. Also dropped -Dlinux too, for the same reason.

Maybe this is what you saw?

Yes, I misunderstood, I'm in fact with v1912 now.

BTW, how can I specify the installation prefix? Is it $WM_PROJECT_DIR? I believe $WM_PROJECT_DIR needs to be set to the root of the extracted source code tarball... so, how can you specify a different installation prefix?

Thanks!!

Very easy - don't do anything! The FOAM_INST_DIR became obsolete starting with 1812. In fact, you can name things pretty much how you want. Even the project directory itself (Eg, /opt/my-openfoam-for-osx/).

The only minor thing to know about is how you want to handle ThirdParty directories. Install with version-specific naming, API-specific naming, use a common installation, or toss it directly into the project directory.

The top-level readme has much more info.

Do you mean that I can move the resulting binaries to any other directory I wish after building them, or that I need to extract the source exactly into the final installation location? (I hope I can move it after the build, because I'm using a ramdisk for compiling)

BTW, something is going wrong because it's trying to use the wmake rules for darwin64Gcc instead of darwin64Clang:

/Volumes/OpenFOAMramdisk/OpenFOAM-v1912/wmake/rules/General/general:33: /Volumes/OpenFOAMramdisk/OpenFOAM-v1912/wmake/rules/darwin64Gcc/general: No such file or directory
make: Failed to remake makefile `/Volumes/OpenFOAMramdisk/OpenFOAM-v1912/wmake/rules/darwin64Gcc/general'.
lemon.c -o /Volumes/OpenFOAMramdisk/OpenFOAM-v1912/wmake/platforms/darwin64Gcc/lemon
make: lemon.c: No such file or directory

And it is strange because it successfully found my custom LLVM build:

./Allwmake 
gcc=/usr/bin/gcc
clang=/Users/cesss/LLVM/7.0.0/bin/clang
mpirun=/Users/cesss/OpenMPI/4.0.2/bin/mpirun
make=/usr/bin/make
cmake=/Users/cesss/CMake/3.15.5/bin/cmake
wmake=/Volumes/OpenFOAMramdisk/OpenFOAM-v1912/wmake/wmake
m4=/usr/bin/m4
flex=/usr/bin/flex

compiler=/Users/cesss/LLVM/7.0.0/bin/clang
clang version 7.0.0 (tags/RELEASE_700/final)

========================================
2020-02-20 13:53:43 +0100
Starting compile OpenFOAM-v1912 Allwmake
  Gcc system compiler
  darwin64GccDPInt32Opt, with SYSTEMOPENMPI openmpi-system
========================================

BTW, I didn't tell wmake about the location of the (already built) dependencies (CGAL, etc). I'm trying to figure out the options for specifying the paths (I built them on my own, similar to the spack way of doing things, but on my own).

Thanks a lot!!

In prefs.sh or bashrc did you tell it to use WM_COMPILER=Clang and source the bashrc after that?

Try with wmake -show-cxx, what -show-path-cxx to see what it is using. Your log file reports which version of tools, but it does have this:

Starting compile OpenFOAM-v1912 Allwmake
  Gcc system compiler
  darwin64GccDPInt32Opt, with SYSTEMOPENMPI openmpi-system

Which means that something has not been configured, or something was forgotten .

A cheap (slightly brutal) way to specify paths is to use the ./bin/tools/foamConfigurePaths script.

It can only be run from the top-level project directory (for some safety). See for example,

https://salsa.debian.org/science-team/openfoam/blob/master/debian/rules#L68

Same thing that I'm doing in the RPM spec file: https://build.opensuse.org/package/view_file/science/openfoam1912/openfoam1912.spec?expand=1

In fact there I'm also using it to overwrite the project directory after installation (in the %post section) since the RPM has been created to be relocatable.

Thanks a lot, @mark , I think I'm progressing. The point where I'm stuck now is that I'm not able to pass my $(CPPFLAGS), $(CXXFLAGS), and $(LDFLAGS) to Allwmake. I need to do that, because I need to specify where my system headers are (-isysroot flag) and the MacOS version to build for (-mmacosx-version-min). By looking at the wmake rules, I saw you use $(GFLAGS), so I tried to put everything into $(GFLAGS), but either $(GFLAGS) is overwritten somewhere, or it's not used for building lemon.

BTW, these are all the steps I'm doing. Note that I built CGAL with my GMP and MPFR, so I'm trying to reenable them, as you'll see in my steps.

In case you notice anything that I should be doing in another way, please tell:

cd /Volumes/ramdisk/OpenFOAM-v1912

cp ~/myeditedbashrc_for_choosingclang/bashrc etc/

export GFLAGS="$CPPFLAGS $LDFLAGS $CXXFLAGS"

source etc/bashrc

bin/tools/foamConfigurePaths -boost boost_1.64.0 -boost-path ~/boost/1.64.0 -cgal cgal_4.9.1 -cgal-path ~/CGAL/4.9.1 -fftw fftw_3.3.8 -fftw-path ~/fftw/3.3.8 -metis metis_5.1.0 -metis-path ~/metis/5.1.0 -scotch scotch_6.0.9 -scotch-path ~/scotch/6.0.9

mv wmake/rules/darwin64Clang/CGAL wmake/rules/darwin64Clang/usegeneralCGALinstead

export GMP_ARCH_PATH=~/gmp/6.1.0

export MPFR_ARCH_PATH=~/mpfr/4.0.2

./Allwmake

Thanks a lot for the great help you are giving me!! I appreciate it very much!!

I hope that your foamConfigurePaths command and your exports are actually using absolute values!

If you specify something like -boost-path $HOME/boost/1.64.0 the shell will expand the $HOME. Not sure if the ~/ will always be expanded before being seen by the command. You will need to examine the etc/config.sh/ files to see what it did.

You will also notice from the files, or from reading the foamConfigurePaths -help that specifying the -boost-path will write the path directly. You can specify what you want for -boost but won't be used in that combination.

If you need to pass in extra flags, could see if this type of change does the trick: 2892b78d

Only the modifications under wmake/rules/ are what you need. Everything else is just extra tidiness (for release) that you don't need.

changed title from interfaceCompositionModel or InterfaceCompositionModel? Or both? Typo or feature? to Case sensitive filesystems, Darwin builds

BTW: changed the title since that's actually what we are dealing with.

Yes, @mark , I'm using full pathnames, I just wrote here ~/ for the sake of simplicity, but actually I'm not using that shortcut in my actual steps.

Thanks a lot, really. I'll need to continue tomorrow because this afternoon I got more tasks than I expected, so I've been unable to try more things. I hope to get it built tomorrow, we'll see :-)

Thanks a million!!

mentioned in commit 5ba2cbc5

Update:

First, your 2892b78d change works great, but I also had to modify wmake/rules/darwin64Clang/c because there's some C source code (lemon for example) so I need to set flags for it too.

Now, I faced a C++ instantiation problem in src/OpenFOAM/containers/HashTables/HashOps/HashOps.C:

/Users/cess/LLVM/7.0.0/include/c++/v1/iterator:457:29: error: no type named 'pointer' in
      'Foam::HashTable<Foam::zero::null, int, Foam::Hash<Foam::label> >::key_iterator_base<Foam::HashTable<Foam::zero::null, int, Foam::Hash<Foam::label>
      >::const_iterator>'
    typedef typename _Iter::pointer           pointer;
            ~~~~~~~~~~~~~~~~^~~~~~~
/Users/cess/LLVM/7.0.0/include/c++/v1/iterator:466:8: note: in instantiation of template class
      'std::__1::__iterator_traits_impl<Foam::HashTable<Foam::zero::null, int, Foam::Hash<Foam::label> >::key_iterator_base<Foam::HashTable<Foam::zero::null, int,
      Foam::Hash<Foam::label> >::const_iterator>, true>' requested here
    :  __iterator_traits_impl
       ^
/Users/cess/LLVM/7.0.0/include/c++/v1/iterator:481:7: note: in instantiation of template class
      'std::__1::__iterator_traits<Foam::HashTable<Foam::zero::null, int, Foam::Hash<Foam::label> >::key_iterator_base<Foam::HashTable<Foam::zero::null, int,
      Foam::Hash<Foam::label> >::const_iterator>, true>' requested here
    : __iterator_traits<_Iter, __has_iterator_category<_Iter>::value> {};
      ^
/Users/cess/LLVM/7.0.0/include/c++/v1/algorithm:2617:31: note: in instantiation of template class
      'std::__1::iterator_traits<Foam::HashTable<Foam::zero::null, int, Foam::Hash<Foam::label> >::key_iterator_base<Foam::HashTable<Foam::zero::null, int,
      Foam::Hash<Foam::label> >::const_iterator> >' requested here
              __less<typename iterator_traits<_ForwardIterator>::value_type>());
                              ^
containers/HashTables/HashOps/HashOps.C:83:27: note: in instantiation of function template specialization 'std::__1::max_element<Foam::HashTable<Foam::zero::null,
      int, Foam::Hash<Foam::label> >::key_iterator_base<Foam::HashTable<Foam::zero::null, int, Foam::Hash<Foam::label> >::const_iterator> >' requested here
    auto const max = std::max_element(locations.begin(), locations.end());
                          ^
1 error generated.
make: *** [/Volumes/OpenFOAMramdisk/OpenFOAM-v1912/build/darwin64ClangDPInt32Opt/src/OpenFOAM/containers/HashTables/HashOps/HashOps.o] Error 1

I'm not an expert at the standard C++ template library, so I don't know really what's going on, but maybe this StackOverflow question might be related.

Again, thanks a lot!!

Funny how different compilers complain about different things. No problem with that bit of code on the linux clang compilers, except for one test case.

However, your problem can probably be fixed with these changes: 8e63f084.

There are a number of them, but the only ones relevant for you would be this bit from HashTable.H, I believe:

@@ -171,10 +170,17 @@ public:
         //- The third template parameter, the hash index method.
         typedef Hash hasher;
 
+        //- Pointer type for storing into value_type objects.
+        //  This type is usually 'value_type*'.
+        typedef T* pointer;
+
         //- Reference to the stored value_type.
         //  This type is usually 'value_type&'.
         typedef T& reference;
 
+        //- Const pointer type for the stored value_type.
+        typedef const T* const_pointer;
+
         //- Const reference to the stored value_type.
         typedef const T& const_reference;

What did you need to change for compiling Lemon?

Case sensitive filesystems, Darwin builds

Child items

Activity

Admin message

Case sensitive filesystems, Darwin builds

Child items

Linked items

Activity