Commit ac4ed6d2 authored by Roger Almenar's avatar Roger Almenar Committed by mark

Initial commit of legacyCompile

parents
Script for installation of OpenFOAM and related ThirdParty packages
===================================================================
This script has been created to ease compiling OpenFOAM with various
combinations of ThirdParty packages.
It is named legacyCompile, since this represents a common situation
in which the standard system toolchain is a bit out of date.
During the compilation, all output messages will be shown on screen as well
as saved in log files. You will find all log files under the `log/` folder.
Although the script has been tested with a variety of OS (mostly Linux)
and package versions, it may not work for all users and environments.
**So use it at your own risk**
---
## Prerequisites
1) The legacyCompile script needs to have execution rights and be
configured according to the package versions desired.
For configuration, please see below.
2) All packages, including OpenFOAM, ThirdParty, and those packs
related to ThirdParty need to be stored under the `packages/`
subdirectory.
3) Names of all packaged files must be as defined under default or
under user-preference. For example, if under user-preference you
have mesa-10.3.2, the file name should NOT be MesaLib-10.3.2.tgz.
The same may apply to the following:
* MesaLib-x.y.z files (replace by mesa-x.y.z)
* llvm-x.y.z.src TarFile (remove .src part in filename and in subdir)
* cfe-x.y.z.src TarFile must be moved to llvm-x.y.z/tools/clang
* ParaView-v* TarFile (remove -source part in filename and in subdir)
Note: when unpacking ParaView, it may create a subdir with suffix
-source. Please remove this suffix as well and tar the package
again.
* And potentially others, except QT files, which have a
completely different naming anyhow.
Please modify names by hand if required.
4) OpenFOAM and ThirdParty packs need to unpack to a subdirectory with
the same name as the file names (without extension). Hence if you
have ThirdParty-v1612+.tgz, it should not unpack to
ThirdParty-plus/ subdir.
5) If you use `clang` as compiler, it must be the same version as `llvm`.
6) Please make sure that the unpacked OpenFOAM `etc/bashrc` file
contains the correct version name
(if you download from other sources than the standard release files).
---
## Script Configuration
Users need to define the desired packages and versions to use for the
OpenFOAM compilation. The packages and versions need to be input under the
paragraph `USER INPUT (2)`, stating whether the default version is to be
used, or the user needs to use a different version. If both entries have
information, the user-preference will prevail. In any case, a suitable
tar/zip file with packageName-versionNumber needs to be placed under subdir
`packages/`. The value `ompiType` refers to the openmpi type (system|third).
# USER INPUT (2)
# ~~~~~~~~~~~~~~
# Preferred software versions etc. Use 'third' for ThirdParty.
#---------------------+-------------------+-------------------+
# package | default | user-preference |
#---------------------+-------------------+-------------------+
boost boost_1_62_0
cgal CGAL-4.9
fftw fftw-3.3.5
openmpi openmpi-1.10.4
scotch scotch_6.0.3
paraview ParaView-5.0.1
cmake cmake-system cmake-3.5.2
vtk none VTK-7.1.0
mesa none mesa-13.0.1
qt system qt-4.8.7
llvm llvm-3.7.0
gcc system gcc-4.8.5
gmp system gmp-6.1.1
mpfr system mpfr-3.1.5
mpc system mpc-1.0.3
label 32
precision DP
ompiType system third
compilerType system third
compiler Gcc Gcc
#--------------------------------------------------------------
---
## Usage
./legacyCompile [OPTION] OPERATION [...OPERATION]
operations:
-unpack Unpack sources, configure
-third Compile ThirdParty
-foam Compile OpenFOAM
-list List configured versions only
-all Alias for -unpack -third -foam
-rebuild Alias for -third -foam
options:
-purge Purge old directories before unpacking (CAUTION)
-help
Config options:
-int32 | -int64 Label width (default: 32)
-SP | -DP Precision (default: DP)
-sigfpe |-no-sigfpe Activate/deactivate FOAM_SIGFPE handling
The script contains the three essential operations:
1) unpack (the `-unpack` option)
2) compile/recompile ThirdParty packages (the `-third` option)
3) compile/recompile OpenFOAM (the `-foam` option)
The `-all` shortcut is a simple way to do all three together.
The `-rebuild` shortcut just does `-third` and `-foam` (without the
unpack stage).
* To compile first time (from scratch) with default label size and precision:
./legacyCompile -all
* To repeat the same, **overwriting** any existing installation:
./legacyCompile -all -purge
* To unpack all packages and configure some OpenFOAM settings without
compiling:
./legacyCompile -unpack
./legacyCompile -unpack -purge # caution: removes existing installation
To compile with 64 bit label size in single precission:
./legacyCompile -all -int64 -SP
If you change your third-party package selections or the preferred
configuration (SP, DP etc), you will need the `-unpack` operation,
which also includes some configuration operations.
./legacyCompile -unpack
To (re)compile ThirdParty and OpenFOAM individually:
./legacyCompile -unpack -third -purge
(to compile ThirdParty from scratch)
./legacyCompile -foam
(to compile OpenFOAM from existing ThirdParty)
To compile with 32 bit label size in single precission:
./legacyCompile -all -int64 -SP
---
## Notes
The standard compilation of OpenFOAM and ThirdParty (as done by
OpenFOAM itself) will only rebuild the out-of-date parts.
Since the building of individual ThirdParty infrastructure components
(eg, the compiler itself, paraview, etc) are all essentially *forced*
builds, the legacyCompile script uses the log files to detect if an
infrastructure component has already been built or not.
If you experience issues when building such a subcomponent (eg,
building Clang), you should remove the corrsponding logfile(s) before
recompiling with the '-unpack -third` options.
---
2016-12-21
#!/bin/bash
#------------------------------------------------------------------------------
# Script
# legacyCompile
#
# Description
# Simple convenience script to install OpenFOAM on older systems.
# Use ThirdParty compiler, but retain most default software versions.
#
# Adjust the relevant USER INPUT sections below prior to running.
#
# - INPUT (1) has the top-level packaging
# - INPUT (2) has version preferences
#
# License
# GPL v3 or later
# Copyright (C) 2016 Roger Almenar, ESI GmbH.
#------------------------------------------------------------------------------
# ~~~~~~~~~~~~~~
# USER INPUT (1)
# ~~~~~~~~~~~~~~
FoamVersion=v1612+
OFdirName=OpenFOAM-$FoamVersion
TPdirName=ThirdParty-$FoamVersion
# Source package names may differ from the directory names
OFsource=$OFdirName
TPsource=$TPdirName
# Local directories
INSTALL_DIR=$PWD/install
PACKAGE_DIR=$PWD/packages
LOGFILE_DIR=$PWD/log
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# User input continues below at USER INPUT (2)
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#------------------------------------------------------------------------------
_preferences()
{
# 'package default version' lines=>packageDefault, packageVersion variable
local line pkg def val
while read -r line
do
set -- $line; pkg="$1"; def="$2"; val="$3"
if [ "$pkg" = "${pkg#[a-z]}" ]
then
continue # Packages must start with lowercase
elif [ $# -eq 2 -o $# -eq 3 ]
then
val="${3:-$def}" # Use default if not specified
eval "${pkg}Default=\"\$def\""; eval "${pkg}Version=\"\$val\""
else
echo "Bad input while setting versions"; echo " $line"
exit 1
fi
done
}
_preferences <<-USER_INPUT
# ~~~~~~~~~~~~~~
# USER INPUT (2)
# ~~~~~~~~~~~~~~
# Preferred software versions etc. Use 'third' for ThirdParty.
#---------------------+-------------------+-------------------+
# package | default | user-preference |
#---------------------+-------------------+-------------------+
boost boost_1_62_0
cgal CGAL-4.9
fftw fftw-3.3.5
openmpi openmpi-1.10.4
scotch scotch_6.0.3
paraview ParaView-5.0.1
cmake cmake-system cmake-3.5.2
vtk none VTK-7.1.0
mesa none mesa-13.0.1
qt qt-system qt-4.8.7
llvm llvm-3.7.0
gcc system gcc-4.8.5
gmp system gmp-6.1.1
mpfr system mpfr-3.1.5
mpc system mpc-1.0.3
label 32
precision DP
ompiType system third
compilerType system third
compiler Gcc Clang
# -----------------------------------------------------------------------------
USER_INPUT
#
# NO FURTHER EDITING BELOW THIS LINE
#------------------------------------------------------------------------------
# Special treatment for gcc sub-components (needs rethinking)
case "$compilerVersion" in
Gcc | Gcc48)
gccVersion=gcc-4.8.5 # match value in config.sh/compiler
;;
Gcc49)
gccVersion=gcc-4.9.3 # match value in config.sh/compiler
;;
Gcc62)
gccVersion=gcc-6.2.0 # match value in config.sh/compiler
;;
Gcc*)
echo "Error: unconfigured gcc version '$compilerVersion'"
exit 1 # Really exit, since this must be an input error
;;
esac
# -----------------------------------------------------------------------------
# NO FURTHER EDITING BELOW THIS LINE
#------------------------------------------------------------------------------
# Name of qt source directory is a fixed value
qtSource="qt-everywhere-opensource-src-${qtVersion##*-}"
# Change 'system' into 'gmp-system' etc
for check in gcc gmp mpfr mpc
do
if eval test "\$${check}Version = system"
then
eval "${check}Version=${check}-system"
eval "${check}Default=${check}-system"
fi
done
# No gcc/gmp/mpfr/mpc for system compilers or non-Gcc compilers
case "$compilerTypeVersion:$compilerVersion" in
third:Gcc*)
;;
third:Clang* | system:Clang* | system:Gcc | system:Icc | system:*)
# System, or known non-gcc
for check in gcc gmp mpfr mpc
do
eval "unset ${check}Version ${check}Default"
done
;;
*)
echo "Error: unknown $compilerTypeVersion-compiler '$compilerVersion'"
exit 1
;;
esac
# Separator lines
_line()
{
echo " ==================== ==================== ===================="
}
# Various formatting for -list output
# formatted print
_fmt2()
{
local arg1="${1:-????}"
local arg2="${2:-undefined}"
printf " %-20s %-20s\n" "$arg1" "$arg2"
}
# formatted print
# name version default [-nocheck]
_fmt3()
{
local what="${1:-????}"
local arg1="${2:-undefined}"
local arg2="${3:-undefined}"
if [ "$#" -eq 2 ]
then
_fmt2 "$what" "$arg1"
elif [ "$#" -eq 3 -a "$arg1" = "$arg2" ]
then
_fmt2 "$what" "$arg1"
else
printf " %-20s %-20s %-20s\n" "$what" "$arg1" "$arg2"
fi
}
# input: 'package'
# output: 'package' packageDefault packageVersion
_fmtDefVer()
{
local pkg="$1"
eval local "def=\$${pkg}Default"
eval local "val=\$${pkg}Version"
_fmt3 "$pkg" "$def" "$val"
}
#------------------------------------------------------------------------------
usage() {
exec 1>&2
while [ "$#" -ge 1 ]; do echo "$1"; shift; done
cat<<USAGE
usage: ${0##*/} [OPTION] OPERATION [...OPERATION]
operations:
-unpack Unpack sources, configure
-third Compile ThirdParty
-foam Compile OpenFOAM
-list List configured versions only
-all Alias for -unpack -third -foam
-rebuild Alias for -third -foam
options:
-purge Purge old directories before unpacking (CAUTION)
-help
Config options:
-int32 | -int64 Label width (default: ${labelDefault:-?})
-SP | -DP Precision (default: ${precisionDefault:-?})
-sigfpe |-no-sigfpe Activate/deactivate FOAM_SIGFPE handling
=========================
Make and install $OFdirName / $TPdirName
package = $PWD/packages
logdir = $PWD/log
install = $PWD/install
USAGE
exit 1
}
#------------------------------------------------------------------------------
# Report error and exit
die()
{
exec 1>&2
echo
echo "Error: see '${0##*/} -help' for usage"
while [ "$#" -ge 1 ]; do echo " $1"; shift; done
echo
exit 1
}
#------------------------------------------------------------------------------
# Location of the OpenFOAM bashrc
BASHRC_FILE=$INSTALL_DIR/$OFdirName/etc/bashrc
# Various building operations
unset runLIST runUNPACK runTHIRD runFOAM purgeOLDFILES
# Pre-load options
unset optLabel optPrec optFpe
case "$labelVersion" in (32|64) optLabel="-int$labelVersion";; esac
case "$precisionVersion" in (DP|SP) optPrec="-$precisionVersion";; esac
# Parse options
while [ "$#" -gt 0 ]
do
case "$1" in
-h | -help | --help | help)
usage
;;
-unpack) # Stage 1: unpack sources, configure
runUNPACK=true
;;
-purge) # (Stage 1): purge old files before unpacking
purgeOLDFILES=true
;;
-third) # Stage 2: compile ThirdParty
runTHIRD=true
;;
-foam) # Stage 3: compile OpenFOAM
runFOAM=true
;;
-all) # Stage 1-3 (all)
runUNPACK=true
runTHIRD=true
runFOAM=true
;;
-rebuild) # Stage 2-3 (rebuild)
runTHIRD=true
runFOAM=true
;;
-list)
optList=true
;;
-int32 | -int64)
optLabel="$1"
;;
-SP | -DP)
optPrec="$1"
;;
-sigfpe | -no-sigfpe)
optFpe="$1"
;;
*)
die "unknown option/argument: '$1'"
;;
esac
shift
done
[ -n "$runUNPACK$runTHIRD$runFOAM$optList" ] || die "No operations specified"
#------------------------------------------------------------------------------
# Set some of the architecture configuration
configArch="$optLabel $optPrec $optFpe"
if [ "$ompiTypeVersion" != "$ompiTypeDefault" ]
then
configArch="$configArch -openmpi-$ompiTypeVersion"
fi
#------------------------------------------------------------------------------
_listSettings()
{
echo
echo "$OFdirName / $TPdirName"
echo
_fmt3 package default version -nocheck
_line
_fmtDefVer boost
_fmtDefVer cgal
_fmtDefVer fftw
_fmtDefVer openmpi
_fmtDefVer scotch
_fmtDefVer paraview
_fmtDefVer cmake
_fmtDefVer vtk
_fmtDefVer mesa
_fmtDefVer qt
_fmtDefVer ompiType
_fmtDefVer llvm
_fmt3 gcc "${gccDefault:-system}" "${gccVersion:-system}"
_fmt3 gmp "${gmpDefault:-system}" "${gmpVersion:-system}"
_fmt3 mpfr "${mpfrDefault:-system}" "${mpfrVersion:-system}"
_fmt3 mpc "${mpcDefault:-system}" "${mpcVersion:-system}"
_line
echo
echo " configure: $configArch"
echo " compiler: -$compilerTypeVersion $compilerVersion"
echo
}
# _mvdir src-dir dst-dir
# Renames src-dir to dst-dir if src-dir exists and dst-dir does not
_mvdir()
{
local src="$1"
local dst="$2"
if [ -n "$src" -a -d "$src" ] && [ -n "$dst" -a ! -e "$dst" ]
then
# TODO? maybe need special treatment if dst is a link (eg, remove it)
echo "Rename $src/ -> $dst/"
mv "$src" "$dst"
fi
}
# _unpack packageName,
# does not unpack anything ending with '-none' or '-system'
#
# Unpack given package from $PACKAGE_DIR if it can be resolved
# (as tar.gz, tar.bz2, ... file) and if "log.unpack..." sentinel
# log-file does not already exist
_unpack()
{
[ $# -gt 0 ] || return 1
local pkgName="$1" # The package name
local logFile="$LOGFILE_DIR/log.unpack_$pkgName.txt"
local pkgFile
pkgFile="${pkgName##*-}"
if [ "$pkgFile" = none -o "$pkgFile" = system ]
then
return 2
fi
if [ -f "$logFile" ]
then
echo "Unpack $pkgName : log-file indicates it was already done"
return 0
fi
# Check archive types (most compact first)
local fileType # tgz | tar.gz | tar.bz2 | tar.xz
for ext in tar.xz tar.bz2 tar.gz tgz tar zip
do
pkgFile="$PACKAGE_DIR/$pkgName.$ext"
if [ -f "$pkgFile" ]
then
fileType=$ext
break
fi
done
[ -n "$fileType" ] || {
echo "Cannot unpack $pkgName : missing $PACKAGE_DIR/$pkgName.???"
return 2
}
echo "Unpack $pkgName $fileType ..."
case "$fileType" in
tgz | tar.gz)
tar -xzvf $pkgFile > "$logFile.$$" 2>&1
;;
tar.bz2)
tar -xjvf $pkgFile > "$logFile.$$" 2>&1
;;
tar.xz)
tar -xJvf $pkgFile > "$logFile.$$" 2>&1
;;
tar)
tar -xvf $pkgFile > "$logFile.$$" 2>&1
;;
zip)
unzip -v $pkgFile > "$logFile.$$" 2>&1
;;
esac
\mv -f "$logFile.$$" "$logFile"
}
# _unpackIfDifferent packageName defaultVersion
#
# Wrapper for _unpack that checks for the existence of the directory
# and only tries to unpack if there is difference between the packageName
# and its default-version
#
_unpackIfDifferent()
{
[ $# -gt 0 ] || return 2
local pkgName="$1" # The package name
local pkgDeft="$2"
if [ -z "$pkgName" ]
then
return 2
elif [ "$pkgName" = "$pkgDeft" -a -d "$pkgName" ]
then
echo " already exists $pkgName"
return 0
fi
_unpack $pkgName
}
# service routine:
# Call with 3 arguments:
# _ifDifferent -opt version default
#
# pass through '-opt version' if version differs from default
_ifDifferent()
{
[ "$#" -eq 3 -a "$2" != "$3" ] && echo "$1 $2"
}
#
# Unpack files and make any adjustments
#
# side-effect: cd to WM_THIRD_PARTY_DIR
#
unpackAllFiles()
{
# Use the foamConfigurePaths tool to configure versions and paths
local foamConfigure=bin/tools/foamConfigurePaths
echo =================================================================
if [ "${purgeOLDFILES:-false}" = true ]
then