Commit f308aa61 authored by Mark Olesen's avatar Mark Olesen

ENH: update handling of versioning and make control (issue #1010)

- Use the OPENFOAM define (eg, 1806, 1812), which normally corresponds
  to a major release, to define an API level. This remains consistent
  within a release cycle and means that it is possible to manage
  several sub-versions and continue to have a consistent lookup.

  The current API value is updated automatically during the build
  and cached as meta data for later use, even when the wmake/ directory
  is missing or OpenFOAM has not yet be initialized.

  The version information reported on program start or with -help
  usage adjusted to reflect this. The build tag from git now also
  carries the date as being more meaningful to trace than a hash
  value.

- Update etc/bashrc and etc/cshrc to obtain the project directory
  directly instead of via its prefix directory. The value obtained
  corresponds to an absolute path, from which the prefix directory
  can be obtained.

  The combination of these changes removes the reliance on any
  particular directory naming convention.
  For example,

     With an 1812 version (API level):

     WM_PROJECT_VERSION=myVersion

     installed as /some/path/somewhere/openfoam-mySandbox

  This makes the -prefix, -foamInstall, -projectVersion, -version
  values of foamEtcFiles, and similar entries for foamConfigurePaths
  superfluous.

  WM_PROJECT_INST_DIR is no longer required or used

ENH: improve handling and discovery of ThirdParty

- improve the flexibility and reusability of ThirdParty packs to cover
  various standard use cases:

    1. Unpacking initial release tar files with two parallel directories
       - OpenFOAM-v1812/
       - ThirdParty-v1812/

    2. With an adjusted OpenFOAM directory name, for whatever reason
       - OpenFOAM-v1812-myCustom/
       - openfoam-1812-other-info/

    3. Operating with/without ThirdParty directory

  To handle these use cases, the following discovery is used.

  Note PROJECT = the OpenFOAM directory `$WM_PROJECT_DIR`
       PREFIX = the parent directory
       VERSION = `$WM_PROJECT_VERSION`
       API = `$WM_PROJECT_API`, as per `foamEtcFiles -show-api`

   0. PROJECT/ThirdParty
      - for single-directory installations

   1. PREFIX/ThirdParty-VERSION
      - this corresponds to the traditional approach

   2. PREFIX/ThirdParty-vAPI
      - allows for an updated value of VERSION (eg, v1812-myCustom)
        without requiring a renamed ThirdParty. The API value
        would still be '1812' and the original ThirdParty-v1812/
        would be found.

   3. PREFIX/ThirdParty-API
      - this is the same as the previous example, but using an unadorned
        API value. This also makes sense if the chosen version name also
        uses the unadorned API value in its naming
        (eg, 1812-patch190131, 1812.19W03)

   4. PREFIX/ThirdParty-common
      - permits maximum reuse for various versions, but only for
        experienced user who are aware of potential version
        incompatibilities

   Directory existence is checked as is the presence of an Allwmake file
   or a platforms/ directory. This reduces the potential of false positive
   matches and limits the selection to directories that are either
   with sources (has the Allwmake file), or pre-compiled binaries (has
   the platforms/ directory).

   If none of the explored directories are found to be suitable,
   it reverts to using a PROJECT/ThirdParty dummy location since
   this is within the project source tree and can be trusted to
   have no negative side-effects.

ENH: add csh support to foamConfigurePaths

- this removes the previously experienced inconsistence in config file
  contents.

REMOVED: foamExec

- was previously used when switching versions and before the
  bashrc/cshrc discovery logic was added. It is now obsolete.
parent a42236c5
# Do not track build information
build-info
# Do not track time-stamp
time-stamp
# META-INFO
Meta-information is for OpenFOAM internal use only.
Do not rely on any files or any file contents in this directory,
or even the existence of this directory.
The format, content and meaning may be changed at anytime without
notice.
The information is provided here for internal documentation purposes.
## api-info
This file and its contents are to be tracked by git.
- File content (api) generated by wmakeBuildInfo from OPENFOAM define
in `wmake/rules/General/general`
- File content (patch) is manually generated content.
## build-info
This file is *never* to be tracked by git, but may be present in shipped
source archives.
- File content (branch, build) generated by wmakeBuildInfo from git
information and cached from previous wmake (api)
## Content types
### api
- 4-digit year-month (YYMM) integer corresponding to the major
release or in unusual cases an intermediate release.
- Format is year-month, as per `date +%y%m`.
Eg, `1712` for the Dec-2017 release.
### patch
- 6-digit year-month-day (YYMMDD) integer corresponding to a patch-level
for the given **released** API.
Development branches have a patch value of `0`.
- Format is year-month-day, as per `date +%y%m%d`.
- The first release is by definition unpatched, and thus carries
a patch value of `0`. If this release were to be patched the following
day, the patch level would jump accordingly.
The patch value is only meaningful together with the api value.
## Flow of information
Changes in the build information must be reflected in information
available in the final binaries. Conversely, it is necessary for later
distributions to have a record of the same information.
| property | source | saved |
|-----------|---------------------------|------------|
| api | wmake/rules | api-info |
| patch | manual (api-info) | build-info |
| branch | git | build-info |
| build | git | build-info |
The command `wmakeBuildInfo -check` is used to determine if
the saved information needs synchronization. The command
`wmakeBuildInfo -update` preforms the synchronitzation.
## Notes
The saved information is split into two separate files. The `api-info`
contains more permanent information, whereas the `build-info` is more
transient in nature.
----
2018-11-29
api=1811
patch=0
......@@ -24,21 +24,20 @@
# \endcode
#
# The -mode option can also be used when chaining settings.
# For example, in the user ~/.OpenFOAM/<VERSION>/config.sh/compiler
# For example, in the user ~/.OpenFOAM/config.sh/compiler
# \code
# eval $(foamEtcFile -sh -mode=go config.sh/compiler)
# \endcode
#
# Environment
# - WM_PROJECT (unset defaults to OpenFOAM)
# - WM_PROJECT_VERSION (unset defaults to detect from path)
# - WM_PROJECT_SITE (unset defaults to PREFIX/site)
# - WM_PROJECT_SITE (unset defaults to PROJECT/site)
#
# Note
# This script must exist in one of these locations:
# - PREFIX/OpenFOAM-<VERSION>/bin
# - PREFIX/openfoam-<VERSION>/bin
# - PREFIX/openfoam<VERSION>/bin
# This script must exist in the project 'bin' directory
#
# The '-show-api' and '-show-patch' options implement partial logic
# from wmake/wmakeBuildInfo.
# Make sure that any changes there are also reflected here.
#
#-------------------------------------------------------------------------------
printHelp() {
......@@ -52,8 +51,6 @@ options:
-list (-l) List directories or files to be checked
-list-test List (existing) directories or files to be checked
-mode=MODE Any combination of u(user), g(group), o(other)
-prefix=DIR Specify an alternative installation prefix
-version=VER Specify alternative OpenFOAM version (eg, 1712, 1806, ...)
-csh Produce 'source FILE' output for a csh eval
-sh Produce '. FILE' output for a sh eval
-csh-verbose As per -csh, with additional verbosity
......@@ -61,6 +58,9 @@ options:
-config Add config directory prefix for shell type:
with -csh* for a config.csh/ prefix
with -sh* for a config.sh/ prefix
-show-api Print api value from wmake/rules, or meta-info and exit
-show-patch Print patch value from meta-info and exit
-with-api=NUM Specify alternative api value to search with
-quiet (-q) Suppress all normal output
-silent (-s) Suppress stderr, except -csh-verbose, -sh-verbose output
-help Print the usage
......@@ -70,8 +70,8 @@ Locate user/group/other file as per '#includeEtc'
Do not group single character options.
Equivalent options:
| -mode=MODE | -mode MODE | -m MODE
| -prefix=DIR | -prefix DIR | -p DIR
| -version=VER | -version VER | -v VER
| -prefix=DIR | -prefix DIR | -p DIR [obsolete 1812]
| -version=VER | -version VER | -v VER [obsolete 1812]
Exit status
0 when the file is found. Print resolved path to stdout.
......@@ -98,80 +98,71 @@ die()
}
#-------------------------------------------------------------------------------
binDir="${0%/*}" # The bin dir
projectDir="${binDir%/bin}" # The project dir
prefixDir="${projectDir%/*}" # The prefix dir (same as $WM_PROJECT_INST_DIR)
binDir="${0%/*}" # The bin dir
projectDir="$(\cd $(dirname $binDir) && \pwd -L)" # Project dir
# Could not resolve projectDir, prefixDir? (eg, called as ./bin/foamEtcFile)
if [ "$prefixDir" = "$projectDir" ]
then
binDir="$(cd $binDir && pwd -L)"
projectDir="${binDir%/bin}"
prefixDir="${projectDir%/*}"
fi
projectDirName="${projectDir##*/}" # The project directory name
projectVersion="$WM_PROJECT_VERSION" # Empty? - will be treated later
userDir="$HOME/.OpenFOAM" # Hard-coded as per foamVersion.H
userDir="$HOME/.OpenFOAM" # As per foamVersion.H
groupDir="${WM_PROJECT_SITE:-$projectDir/site}" # As per foamVersion.H
#-------------------------------------------------------------------------------
# Guess project version or simply get the stem part of the projectDirName.
# Handle standard naming conventions:
# The API locations. See wmake/wmakeBuildInfo
rulesFile="$projectDir/wmake/rules/General/general"
metaInfoDir="$projectDir/META-INFO"
# Get api from rules/General/general
#
# * OpenFOAM-<version>[-extra...]
# * openfoam-<version>[-extra...]
# * openfoam<digits>
# Failure modes:
# - No api information (can't find file etc).
# -> Fatal for building, but could be OK for a stripped down version
#
# - projectVersion: update unless already set
# Fallback. Get from api-info
#
# Helper variables:
# - dirBase (for reassembling name) == projectDirName without the version
unset dirBase
guessVersion()
getApi()
{
local version
case "$projectDirName" in
(OpenFOAM-* | openfoam-*)
# Dashed naming: OpenFOAM-<VERSION> or openfoam-<VERSION>
dirBase="${projectDirName%%-*}-"
version="${projectDirName#*-}"
version="${version%%*-}" # Extra safety, eg openfoam-version-packager
;;
(openfoam[0-9]*)
# Debian-style naming: openfoam<VERSION>
dirBase="openfoam"
version="${projectDirName#openfoam}"
;;
local value
(*)
die "unknown/unsupported naming convention for '$projectDirName'"
;;
esac
value="$(sed -ne '/^ *#/!{ /WM_VERSION.*OPENFOAM=/{ s@^.*OPENFOAM= *\([0-9][0-9]*\).*@\1@p; q }}' $rulesFile 2>/dev/null)"
if [ -z "$value" ] && [ -f "$metaInfoDir/api-info" ]
then
# Fallback. Get from api-info
value="$(sed -ne 's@^ *api *= *\([0-9][0-9]*\).*@\1@p' $metaInfoDir/api-info 2>/dev/null)"
fi
# Set projectVersion if required
: ${projectVersion:=$version}
if [ -n "$value" ]
then
echo "$value"
else
return 1
fi
}
# Set projectVersion and update versionNum, projectDirName accordingly
setVersion()
# Get patch from meta-info / api-info
#
# Failure modes:
# - No patch information (can't find file etc).
#
getPatchLevel()
{
projectVersion="$1"
local value
# Need dirBase when reassembling projectDirName
[ -n "$dirBase" ] || guessVersion
# Fallback. Get from api-info
value="$(sed -ne 's@^ *patch *= *\([0-9][0-9]*\).*@\1@p' $metaInfoDir/api-info 2>/dev/null)"
projectDirName="$dirBase$projectVersion"
if [ -n "$value" ]
then
echo "$value"
else
return 1
fi
}
#-------------------------------------------------------------------------------
optMode=ugo # Default mode is always 'ugo'
unset shellOutput verboseOutput
unset optAll optConfig optList optVersion
unset optAll optConfig optList projectApi
# Parse options
while [ "$#" -gt 0 ]
......@@ -180,6 +171,19 @@ do
-h | -help*)
printHelp
;;
-show-api)
# Show API and exit
getApi
exit $?
;;
-show-patch)
# Show patch level and exit
getPatchLevel
exit $?
;;
-with-api=*)
projectApi="${1#*=}"
;;
-a | -all)
optAll=true
unset shellOutput verboseOutput
......@@ -204,13 +208,6 @@ do
-mode=[ugo]*)
optMode="${1#*=}"
;;
-prefix=/*)
prefixDir="${1#*=}"
prefixDir="${prefixDir%/}"
;;
-version=*)
optVersion="${1#*=}"
;;
-m | -mode)
optMode="$2"
shift
......@@ -223,22 +220,23 @@ do
;;
esac
;;
-p | -prefix)
[ "$#" -ge 2 ] || die "'$1' option requires an argument"
prefixDir="${2%/}"
shift
;;
-q | -quiet)
optQuiet=true
;;
-s | -silent)
optSilent=true
;;
-v | -version)
-prefix=* | -version=*)
echo "ignored defunct option '${1%%=*}'" 1>&2
;;
-p | -prefix | -v | -version)
# Ignored, but still need to check/discard its argument
[ "$#" -ge 2 ] || die "'$1' option requires an argument"
optVersion="$2"
echo "ignored defunct option '$1'" 1>&2
shift
;;
--)
shift
break
......@@ -255,6 +253,10 @@ done
#-------------------------------------------------------------------------------
# Establish the API value
[ -n "$projectApi" ] || projectApi=$(getApi)
# Split arguments into filename (for searching) and trailing bits for shell eval
# Silently remove leading ~OpenFOAM/ (as per Foam::findEtcFile)
nArgs=$#
......@@ -279,25 +281,9 @@ then
fi
# Get version information
if [ -n "$optVersion" ]
then
setVersion $optVersion
elif [ -z "$projectVersion" ]
then
guessVersion
fi
# Updates:
# - projectDir for changes via -prefix or -version
# - groupDir for changes via -prefix
projectDir="$prefixDir/$projectDirName"
groupDir="${WM_PROJECT_SITE:-$prefixDir/site}"
# Debugging:
# echo "Installed locations:" 1>&2
# for i in projectDir prefixDir projectDirName projectVersion
# for i in projectDir
# do
# eval echo "$i=\$$i" 1>&2
# done
......@@ -305,12 +291,12 @@ groupDir="${WM_PROJECT_SITE:-$prefixDir/site}"
# Define the various places to be searched:
unset dirList
case "$optMode" in (*u*) # (U)ser
dirList="$dirList $userDir/$projectVersion $userDir"
dirList="$dirList $userDir/$projectApi $userDir"
;;
esac
case "$optMode" in (*g*) # (G)roup == site
dirList="$dirList $groupDir/$projectVersion/etc $groupDir/etc"
dirList="$dirList $groupDir/$projectApi/etc $groupDir/etc"
;;
esac
......
#!/bin/bash
#------------------------------------------------------------------------------
# ========= |
# \\ / F ield | OpenFOAM: The Open Source CFD Toolbox
# \\ / O peration |
# \\ / A nd | Copyright (C) 2011-2016 OpenFOAM Foundation
# \\/ M anipulation | Copyright (C) 2018 OpenCFD Ltd.
#-------------------------------------------------------------------------------
# License
# This file is part of OpenFOAM.
#
# OpenFOAM is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with OpenFOAM. If not, see <http://www.gnu.org/licenses/>.
#
# Script
# foamExec
#
# Description
# Usage: foamExec [-version=foamVersion] <foamCommand> ...
#
# Runs the <foamVersion> version of executable <foamCommand>
# with the rest of the arguments.
#
# Can also be used for parallel runs. For example,
# \code
# mpirun -np <nProcs> \
# foamExec -version=VERSION <foamCommand> ... -parallel
# \endcode
#
# Note
# This script must exist in $WM_PROJECT_INST_DIR/OpenFOAM-<VERSION>/bin
# or $WM_PROJECT_INST_DIR/openfoam<VERSION>/bin (debian)
#
# foamEtcFile located in the same directory as this script
#
# See also
# foamEtcFile
#
#------------------------------------------------------------------------------
usage() {
exec 1>&2
while [ "$#" -ge 1 ]; do echo "$1"; shift; done
cat<<USAGE
Usage: ${0##*/} [OPTION] <application> ...
options:
-mode=MODE Any combination of u(user), g(group), o(other)
-prefix=DIR Specify an alternative installation prefix
pass through to foamEtcFile
-version=VER Specify alternative OpenFOAM version (eg, 3.0, 1612, ...)
pass through to foamEtcFile
-help Print the usage
Run a particular OpenFOAM version of <APPLICATION>
USAGE
exit 1
}
#-------------------------------------------------------------------------------
binDir="${0%/*}" # The bin dir
projectDir="${binDir%/bin}" # The project dir
# prefixDir="${projectDir%/*}" # The prefix dir (same as $WM_PROJECT_INST_DIR)
projectVersion="${WM_PROJECT_VERSION:-unknown}"
unset etcOpts
# parse options
while [ "$#" -gt 0 ]
do
case "$1" in
-h | -help*)
usage
;;
-mode=*)
etcOpts="$etcOpts $1" # pass-thru to foamEtcFile
;;
-prefix=/*)
etcOpts="$etcOpts $1" # pass-thru to foamEtcFile
;;
-version=*)
etcOpts="$etcOpts $1" # pass-thru to foamEtcFile
projectVersion="${1#*=}" # for reporting
;;
-m | -mode)
[ "$#" -ge 2 ] || usage "'$1' option requires an argument"
etcOpts="$etcOpts $1 $2" # pass-thru to foamEtcFile
shift
;;
-p | -prefix)
[ "$#" -ge 2 ] || usage "'$1' option requires an argument"
etcOpts="$etcOpts $1 $2" # pass-thru to foamEtcFile
shift
;;
-v | -version)
[ "$#" -ge 2 ] || usage "'$1' option requires an argument"
etcOpts="$etcOpts $1 $2" # pass-thru to foamEtcFile
projectVersion="$2" # for reporting
shift
;;
--)
shift
break
;;
-*)
usage "invalid option '$1'"
;;
*)
break
;;
esac
shift
done
# Find and source the OpenFOAM <etc/bashrc>
# placed in function to preserve command-line arguments
sourceBashrc()
{
rcFile="$($binDir/foamEtcFile $etcOpts bashrc)" || {
echo "Error: bashrc file could not be found for OpenFOAM-$projectVersion" 1>&2
exit 2
}
. $rcFile $FOAM_SETTINGS
}
[ "$#" -ge 1 ] || usage "no application specified"
sourceBashrc
exec "$@"
#------------------------------------------------------------------------------
......@@ -297,9 +297,7 @@ reportExecutable()
checkOpenFOAMEnvironment()
{
[ -d "$WM_PROJECT_INST_DIR" ] && \
[ -d "$WM_PROJECT_DIR" ] && \
[ -d "$WM_THIRD_PARTY_DIR" ] || {
[ -d "$WM_PROJECT_DIR" ] && [ -d "$WM_THIRD_PARTY_DIR" ] || {
echo ""
echo "FATAL ERROR: OpenFOAM environment not configured."
echo ""
......@@ -389,7 +387,6 @@ COL5="Crit"
hline
echo "$COL1 $COL2 $COL3 $COL5"
hline
reportEnv '$WM_PROJECT_INST_DIR' noPath yes
reportEnv '$WM_PROJECT_USER_DIR' noPath no
reportEnv '$WM_THIRD_PARTY_DIR' noPath yes
hline
......@@ -415,11 +412,12 @@ hline
reportEnv '$FOAM_LIBBIN' '$LD_LIBRARY_PATH' yes
reportEnv '$FOAM_SITE_LIBBIN' '$LD_LIBRARY_PATH' no
reportEnv '$FOAM_USER_LIBBIN' '$LD_LIBRARY_PATH' no
reportEnv '$FOAM_EXT_LIBBIN' '$LD_LIBRARY_PATH' maybe
reportEnv '$MPI_ARCH_PATH' '$LD_LIBRARY_PATH' yes
hline
#------------------------------------------------------------------------------
heading "Third-party software"
heading "Software Components"
hline
echo "$(fixlen Software 9) $(fixlen Version 10) $(fixlen Location 10)"
hline
......
......@@ -28,11 +28,16 @@
# Description
# Extract data for each time-step from a log file for graphing.
#
# Environment
# WM_PROJECT_API
# WM_PROJECT_DIR
# WM_PROJECT_SITE
#
#------------------------------------------------------------------------------
Script=${0##*/}
toolsDir=${0%/*}/tools
siteDir="${WM_PROJECT_SITE:-${WM_PROJECT_DIR:-<unknown>}/site}"
userDir=$HOME/.OpenFOAM
Script="${0##*/}"
toolsDir="${0%/*}/tools"
groupDir="${WM_PROJECT_SITE:-${WM_PROJECT_DIR:-<unknown>}/site}"
userDir="$HOME/.OpenFOAM"
usage() {
exec 1>&2
......@@ -85,11 +90,11 @@ cat <<HELP
The database ($Script.db) will taken from these locations:
.
$userDir/$WM_PROJECT_VERSION
$userDir
$siteDir/$WM_PROJECT_VERSION
$siteDir
$WM_PROJECT_DIR/etc
$userDir/$WM_PROJECT_API/
$userDir/
$groupDir/$WM_PROJECT_API/etc/
$groupDir/etc/
$WM_PROJECT_DIR/etc/
$toolsDir
option -quiet : suppresses the default information and only prints the
......
......@@ -4,7 +4,7 @@
# \\ / F ield | OpenFOAM: The Open Source CFD Toolbox
# \\ / O peration |
# \\ / A nd | Copyright (C) 2011 OpenFOAM Foundation
# \\/ M anipulation |
# \\/ M anipulation | Copyright (C) 2018 OpenCFD Ltd.
#-------------------------------------------------------------------------------
# License
# This file is part of OpenFOAM.
......@@ -29,10 +29,15 @@
# Create a new case from a template for particular applications
# - requires rsync
#
# Environment
# WM_PROJECT_API
# WM_PROJECT_DIR
# WM_PROJECT_SITE
#
#------------------------------------------------------------------------------
siteDir="${WM_PROJECT_SITE:-${WM_PROJECT_DIR:-<unknown>}/site}"
groupDir="${WM_PROJECT_SITE:-${WM_PROJECT_DIR:-<unknown>}/site}"
userDir="$HOME/.OpenFOAM"
version="${WM_PROJECT_VERSION:-unknown}"
projectApi="${WM_PROJECT_API:-unknown}"
templateDir="appTemplates"
......@@ -44,20 +49,22 @@ usage() {
Usage: ${0##*/} [OPTION]
options:
-app <name> specify the application to use
-case <dir> specify alternative case directory, default is the cwd
-app NAME specify the application to use
-case DIR specify alternative case directory, default is the cwd
-list list the applications available
-version <ver> specify an alternative version (default: '$WM_PROJECT_VERSION')
-with-api=NUM specify alternative api to use (default: \$WM_PROJECT_API)
-version VER [obsolete]
-help Print the usage
clone initial application settings to the specified case from
$userDir/$templateDir/{$version,}/<APP>
$siteDir/$templateDir/{$version,}/<APP>
$userDir/$templateDir/{$projectApi,}/APP
$groupDir/$templateDir/{$projectApi,}/APP
USAGE
exit 1
}
#------------------------------------------------------------------------------
unset appName caseName listOpt
unset appName caseName optList
# parse options