ENH: update handling of versioning and make control (issue #1010)

- Use the OPENFOAM define (eg, 1806, 1812), which normally corresponds
  to a major release, to define an API level. This remains consistent
  within a release cycle and means that it is possible to manage
  several sub-versions and continue to have a consistent lookup.

  The current API value is updated automatically during the build
  and cached as meta data for later use, even when the wmake/ directory
  is missing or OpenFOAM has not yet be initialized.

  The version information reported on program start or with -help
  usage adjusted to reflect this. The build tag from git now also
  carries the date as being more meaningful to trace than a hash
  value.

- Update etc/bashrc and etc/cshrc to obtain the project directory
  directly instead of via its prefix directory. The value obtained
  corresponds to an absolute path, from which the prefix directory
  can be obtained.

  The combination of these changes removes the reliance on any
...

META-INFO/.gitignore

+# Do not track build information
+build-info
+
+# Do not track time-stamp
+time-stamp

META-INFO/README.md

+# META-INFO
+
+Meta-information is for OpenFOAM internal use only.
+
+Do not rely on any files or any file contents in this directory,
+or even the existence of this directory.
+
+The format, content and meaning may be changed at anytime without
+notice.
+
+The information is provided here for internal documentation purposes.
+
+## api-info
+
+This file and its contents are to be tracked by git.
+
+- File content (api) generated by wmakeBuildInfo from OPENFOAM define
+  in `wmake/rules/General/general`
+
+- File content (patch) is manually generated content.
+
+
+## build-info
+
+This file is *never* to be tracked by git, but may be present in shipped
+source archives.
+
+- File content (branch, build) generated by wmakeBuildInfo from git
+  information and cached from previous wmake (api)
+
+
+## Content types
+
+### api
+
+- 4-digit year-month (YYMM) integer corresponding to the major
+  release or in unusual cases an intermediate release.
+
+- Format is year-month, as per `date +%y%m`.
+  Eg, `1712` for the Dec-2017 release.
+
+
+### patch
+
+- 6-digit year-month-day (YYMMDD) integer corresponding to a patch-level
+  for the given **released** API.
+  Development branches have a patch value of `0`.
+
+- Format is year-month-day, as per `date +%y%m%d`.
+
+- The first release is by definition unpatched, and thus carries
+  a patch value of `0`. If this release were to be patched the following
+  day, the patch level would jump accordingly.
+
+The patch value is only meaningful together with the api value.
+
+
+## Flow of information
+
+Changes in the build information must be reflected in information
+available in the final binaries. Conversely, it is necessary for later
+distributions to have a record of the same information.
+
+| property  | source                    | saved      |
+|-----------|---------------------------|------------|
+| api       | wmake/rules               | api-info   |
+| patch     | manual (api-info)         | build-info |
+| branch    | git                       | build-info |
+| build     | git                       | build-info |
+
+
+The command `wmakeBuildInfo -check` is used to determine if
+the saved information needs synchronization. The command
+`wmakeBuildInfo -update` preforms the synchronitzation.
+
+
+## Notes
+
+The saved information is split into two separate files. The `api-info`
+contains more permanent information, whereas the `build-info` is more
+transient in nature.
+
+----
+2018-11-29

META-INFO/api-info

+api=1811
+patch=0

bin/foamEtcFile

@@ -24,21 +24,20 @@
 #     \endcode
 #
 #     The -mode option can also be used when chaining settings.
-#     For example, in the user ~/.OpenFOAM/<VERSION>/config.sh/compiler
+#     For example, in the user ~/.OpenFOAM/config.sh/compiler
 #     \code
 #        eval $(foamEtcFile -sh -mode=go config.sh/compiler)
 #     \endcode
 #
 # Environment
-#     - WM_PROJECT              (unset defaults to OpenFOAM)
-#     - WM_PROJECT_VERSION      (unset defaults to detect from path)
-#     - WM_PROJECT_SITE         (unset defaults to PREFIX/site)
+#     - WM_PROJECT_SITE         (unset defaults to PROJECT/site)
 #
 # Note
-#     This script must exist in one of these locations:
-#     - PREFIX/OpenFOAM-<VERSION>/bin
-#     - PREFIX/openfoam-<VERSION>/bin
-#     - PREFIX/openfoam<VERSION>/bin
+#     This script must exist in the project 'bin' directory
+#
+#     The '-show-api' and '-show-patch' options implement partial logic
+#     from wmake/wmakeBuildInfo.
+#     Make sure that any changes there are also reflected here.
 #
 #-------------------------------------------------------------------------------
 printHelp() {
@@ -52,8 +51,6 @@ options:
   -list (-l)        List directories or files to be checked
   -list-test        List (existing) directories or files to be checked
   -mode=MODE        Any combination of u(user), g(group), o(other)
-  -prefix=DIR       Specify an alternative installation prefix
-  -version=VER      Specify alternative OpenFOAM version (eg, 1712, 1806, ...)
   -csh              Produce 'source FILE' output for a csh eval
   -sh               Produce '. FILE' output for a sh eval
   -csh-verbose      As per -csh, with additional verbosity
@@ -61,6 +58,9 @@ options:
   -config           Add config directory prefix for shell type:
                         with -csh* for a config.csh/ prefix
                         with -sh*  for a config.sh/ prefix
+  -show-api         Print api value from wmake/rules, or meta-info and exit
+  -show-patch       Print patch value from meta-info and exit
+  -with-api=NUM     Specify alternative api value to search with
   -quiet (-q)       Suppress all normal output
   -silent (-s)      Suppress stderr, except -csh-verbose, -sh-verbose output
   -help             Print the usage
@@ -70,8 +70,8 @@ Locate user/group/other file as per '#includeEtc'
 Do not group single character options.
 Equivalent options:
   |  -mode=MODE     |  -mode MODE     | -m MODE
-  |  -prefix=DIR    |  -prefix DIR    | -p DIR
-  |  -version=VER   |  -version VER   | -v VER
+  |  -prefix=DIR    |  -prefix DIR    | -p DIR   [obsolete 1812]
+  |  -version=VER   |  -version VER   | -v VER   [obsolete 1812]
 
 Exit status
     0  when the file is found. Print resolved path to stdout.
@@ -98,80 +98,71 @@ die()
 }
 
 #-------------------------------------------------------------------------------
-binDir="${0%/*}"                # The bin dir
-projectDir="${binDir%/bin}"     # The project dir
-prefixDir="${projectDir%/*}"    # The prefix dir (same as $WM_PROJECT_INST_DIR)
+binDir="${0%/*}"                                    # The bin dir
+projectDir="$(\cd $(dirname $binDir) && \pwd -L)"   # Project dir
 
-# Could not resolve projectDir, prefixDir? (eg, called as ./bin/foamEtcFile)
-if [ "$prefixDir" = "$projectDir" ]
-then
-    binDir="$(cd $binDir && pwd -L)"
-    projectDir="${binDir%/bin}"
-    prefixDir="${projectDir%/*}"
-fi
-projectDirName="${projectDir##*/}"      # The project directory name
-
-projectVersion="$WM_PROJECT_VERSION"    # Empty? - will be treated later
-userDir="$HOME/.OpenFOAM"               # Hard-coded as per foamVersion.H
+userDir="$HOME/.OpenFOAM"                           # As per foamVersion.H
+groupDir="${WM_PROJECT_SITE:-$projectDir/site}"     # As per foamVersion.H
 
 #-------------------------------------------------------------------------------
 
-# Guess project version or simply get the stem part of the projectDirName.
-# Handle standard naming conventions:
+# The API locations. See wmake/wmakeBuildInfo
+rulesFile="$projectDir/wmake/rules/General/general"
+metaInfoDir="$projectDir/META-INFO"
+
+# Get api from rules/General/general
 #
-# * OpenFOAM-<version>[-extra...]
-# * openfoam-<version>[-extra...]
-# * openfoam<digits>
+# Failure modes:
+# - No api information (can't find file etc).
+#   -> Fatal for building, but could be OK for a stripped down version
 #
-# - projectVersion: update unless already set
+# Fallback. Get from api-info
 #
-# Helper variables:
-# - dirBase (for reassembling name) == projectDirName without the version
-unset dirBase
-guessVersion()
+getApi()
 {
-    local version
-
-    case "$projectDirName" in
-    (OpenFOAM-* | openfoam-*)
-        # Dashed naming: OpenFOAM-<VERSION> or openfoam-<VERSION>
-        dirBase="${projectDirName%%-*}-"
-        version="${projectDirName#*-}"
-        version="${version%%*-}" # Extra safety, eg openfoam-version-packager
-        ;;
-
-    (openfoam[0-9]*)
-        # Debian-style naming: openfoam<VERSION>
-        dirBase="openfoam"
-        version="${projectDirName#openfoam}"
-        ;;
+    local value
 
-    (*)
-        die "unknown/unsupported naming convention for '$projectDirName'"
-        ;;
-    esac
+    value="$(sed -ne '/^ *#/!{ /WM_VERSION.*OPENFOAM=/{ s@^.*OPENFOAM= *\([0-9][0-9]*\).*@\1@p; q }}' $rulesFile 2>/dev/null)"
+    if [ -z "$value" ] && [ -f "$metaInfoDir/api-info" ]
+    then
+        # Fallback. Get from api-info
+        value="$(sed -ne 's@^ *api *= *\([0-9][0-9]*\).*@\1@p' $metaInfoDir/api-info 2>/dev/null)"
+    fi
 
-    # Set projectVersion if required
-    : ${projectVersion:=$version}
+    if [ -n "$value" ]
+    then
+        echo "$value"
+    else
+        return 1
+    fi
 }
 
 
-# Set projectVersion and update versionNum, projectDirName accordingly
-setVersion()
+# Get patch from meta-info / api-info
+#
+# Failure modes:
+# - No patch information (can't find file etc).
+#
+getPatchLevel()
 {
-    projectVersion="$1"
+    local value
 
-    # Need dirBase when reassembling projectDirName
-    [ -n "$dirBase" ] || guessVersion
+    # Fallback. Get from api-info
+    value="$(sed -ne 's@^ *patch *= *\([0-9][0-9]*\).*@\1@p' $metaInfoDir/api-info 2>/dev/null)"
 
-    projectDirName="$dirBase$projectVersion"
+    if [ -n "$value" ]
+    then
+        echo "$value"
+    else
+        return 1
+    fi
 }
 
 
 #-------------------------------------------------------------------------------
 optMode=ugo         # Default mode is always 'ugo'
 unset shellOutput verboseOutput
-unset optAll optConfig optList optVersion
+unset optAll optConfig optList projectApi
 
 # Parse options
 while [ "$#" -gt 0 ]
@@ -180,6 +171,19 @@ do
     -h | -help*)
         printHelp
         ;;
+    -show-api)
+        # Show API and exit
+        getApi
+        exit $?
+        ;;
+    -show-patch)
+        # Show patch level and exit
+        getPatchLevel
+        exit $?
+        ;;
+    -with-api=*)
+        projectApi="${1#*=}"
+        ;;
     -a | -all)
         optAll=true
         unset shellOutput verboseOutput
@@ -204,13 +208,6 @@ do
     -mode=[ugo]*)
         optMode="${1#*=}"
         ;;
-    -prefix=/*)
-        prefixDir="${1#*=}"
-        prefixDir="${prefixDir%/}"
-        ;;
-    -version=*)
-        optVersion="${1#*=}"
-        ;;
     -m | -mode)
         optMode="$2"
         shift
@@ -223,22 +220,23 @@ do
             ;;
         esac
         ;;
-    -p | -prefix)
-        [ "$#" -ge 2 ] || die "'$1' option requires an argument"
-        prefixDir="${2%/}"
-        shift
-        ;;
     -q | -quiet)
         optQuiet=true
         ;;
     -s | -silent)
         optSilent=true
         ;;
-    -v | -version)
+
+    -prefix=* | -version=*)
+        echo "ignored defunct option '${1%%=*}'" 1>&2
+        ;;
+    -p | -prefix | -v | -version)
+        # Ignored, but still need to check/discard its argument
         [ "$#" -ge 2 ] || die "'$1' option requires an argument"
-        optVersion="$2"
+        echo "ignored defunct option '$1'" 1>&2
         shift
         ;;
+
     --)
         shift
         break
@@ -255,6 +253,10 @@ done
 
 #-------------------------------------------------------------------------------
 
+# Establish the API value
+[ -n "$projectApi" ] || projectApi=$(getApi)
+
+
 # Split arguments into filename (for searching) and trailing bits for shell eval
 # Silently remove leading ~OpenFOAM/ (as per Foam::findEtcFile)
 nArgs=$#
@@ -279,25 +281,9 @@ then
 fi
 
 
-# Get version information
-if [ -n "$optVersion" ]
-then
-    setVersion $optVersion
-elif [ -z "$projectVersion" ]
-then
-    guessVersion
-fi
-
-# Updates:
-# - projectDir  for changes via -prefix or -version
-# - groupDir    for changes via -prefix
-projectDir="$prefixDir/$projectDirName"
-groupDir="${WM_PROJECT_SITE:-$prefixDir/site}"
-
-
 # Debugging:
 # echo "Installed locations:" 1>&2
-# for i in projectDir prefixDir projectDirName projectVersion
+# for i in projectDir
 # do
 #     eval echo "$i=\$$i" 1>&2
 # done
@@ -305,12 +291,12 @@ groupDir="${WM_PROJECT_SITE:-$prefixDir/site}"
 # Define the various places to be searched:
 unset dirList
 case "$optMode" in (*u*) # (U)ser
-    dirList="$dirList $userDir/$projectVersion $userDir"
+    dirList="$dirList $userDir/$projectApi $userDir"
     ;;
 esac
 
 case "$optMode" in (*g*) # (G)roup == site
-    dirList="$dirList $groupDir/$projectVersion/etc $groupDir/etc"
+    dirList="$dirList $groupDir/$projectApi/etc $groupDir/etc"
     ;;
 esac
 

bin/foamExec

-#!/bin/bash
-#------------------------------------------------------------------------------
-# =========                 |
-# \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
-#  \\    /   O peration     |
-#   \\  /    A nd           | Copyright (C) 2011-2016 OpenFOAM Foundation
-#    \\/     M anipulation  | Copyright (C) 2018 OpenCFD Ltd.
-#-------------------------------------------------------------------------------
-# License
-#     This file is part of OpenFOAM.
-#
-#     OpenFOAM is free software: you can redistribute it and/or modify it
-#     under the terms of the GNU General Public License as published by
-#     the Free Software Foundation, either version 3 of the License, or
-#     (at your option) any later version.
-#
-#     OpenFOAM is distributed in the hope that it will be useful, but WITHOUT
-#     ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
-#     FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
-#     for more details.
-#
-#     You should have received a copy of the GNU General Public License
-#     along with OpenFOAM.  If not, see <http://www.gnu.org/licenses/>.
-#
-# Script
-#     foamExec
-#
-# Description
-#     Usage: foamExec [-version=foamVersion] <foamCommand> ...
-#
-#     Runs the <foamVersion> version of executable <foamCommand>
-#     with the rest of the arguments.
-#
-#     Can also be used for parallel runs. For example,
-#     \code
-#         mpirun -np <nProcs> \
-#             foamExec -version=VERSION <foamCommand> ... -parallel
-#     \endcode
-#
-# Note
-#     This script must exist in $WM_PROJECT_INST_DIR/OpenFOAM-<VERSION>/bin
-#     or $WM_PROJECT_INST_DIR/openfoam<VERSION>/bin (debian)
-#
-#     foamEtcFile located in the same directory as this script
-#
-# See also
-#    foamEtcFile
-#
-#------------------------------------------------------------------------------
-usage() {
-    exec 1>&2
-    while [ "$#" -ge 1 ]; do echo "$1"; shift; done
-    cat<<USAGE
-
-Usage: ${0##*/} [OPTION] <application> ...
-
-options:
-  -mode=MODE        Any combination of u(user), g(group), o(other)
-  -prefix=DIR       Specify an alternative installation prefix
-                    pass through to foamEtcFile
-  -version=VER      Specify alternative OpenFOAM version (eg, 3.0, 1612, ...)
-                    pass through to foamEtcFile
-  -help             Print the usage
-
-Run a particular OpenFOAM version of <APPLICATION>
-
-USAGE
-    exit 1
-}
-
-#-------------------------------------------------------------------------------
-binDir="${0%/*}"                # The bin dir
-projectDir="${binDir%/bin}"     # The project dir
-# prefixDir="${projectDir%/*}"    # The prefix dir (same as $WM_PROJECT_INST_DIR)
-
-projectVersion="${WM_PROJECT_VERSION:-unknown}"
-
-unset etcOpts
-# parse options
-while [ "$#" -gt 0 ]
-do
-    case "$1" in
-    -h | -help*)
-        usage
-        ;;
-    -mode=*)
-        etcOpts="$etcOpts $1"       # pass-thru to foamEtcFile
-        ;;
-    -prefix=/*)
-        etcOpts="$etcOpts $1"       # pass-thru to foamEtcFile
-        ;;
-    -version=*)
-        etcOpts="$etcOpts $1"       # pass-thru to foamEtcFile
-        projectVersion="${1#*=}"    # for reporting
-        ;;
-    -m | -mode)
-        [ "$#" -ge 2 ] || usage "'$1' option requires an argument"
-        etcOpts="$etcOpts $1 $2"    # pass-thru to foamEtcFile
-        shift
-        ;;
-    -p | -prefix)
-        [ "$#" -ge 2 ] || usage "'$1' option requires an argument"
-        etcOpts="$etcOpts $1 $2"    # pass-thru to foamEtcFile
-        shift
-        ;;
-    -v | -version)
-        [ "$#" -ge 2 ] || usage "'$1' option requires an argument"
-        etcOpts="$etcOpts $1 $2"    # pass-thru to foamEtcFile
-        projectVersion="$2"         # for reporting
-        shift
-        ;;
-    --)
-        shift
-        break
-        ;;
-    -*)
-        usage "invalid option '$1'"
-        ;;
-    *)
-        break
-        ;;
-    esac
-    shift
-done
-
-
-# Find and source the OpenFOAM <etc/bashrc>
-# placed in function to preserve command-line arguments
-sourceBashrc()
-{
-    rcFile="$($binDir/foamEtcFile $etcOpts bashrc)" || {
-        echo "Error: bashrc file could not be found for OpenFOAM-$projectVersion" 1>&2
-        exit 2
-    }
-
-    . $rcFile $FOAM_SETTINGS
-}
-
-
-[ "$#" -ge 1 ] || usage "no application specified"
-
-sourceBashrc
-exec "$@"
-
-#------------------------------------------------------------------------------

bin/foamInstallationTest

@@ -297,9 +297,7 @@ reportExecutable()
 
 checkOpenFOAMEnvironment()
 {
-    [ -d "$WM_PROJECT_INST_DIR" ] && \
-    [ -d "$WM_PROJECT_DIR" ] && \
-    [ -d "$WM_THIRD_PARTY_DIR" ] || {
+    [ -d "$WM_PROJECT_DIR" ] && [ -d "$WM_THIRD_PARTY_DIR" ] || {
         echo ""
         echo "FATAL ERROR: OpenFOAM environment not configured."
         echo ""
@@ -389,7 +387,6 @@ COL5="Crit"
 hline
 echo "$COL1 $COL2 $COL3      $COL5"
 hline
-reportEnv '$WM_PROJECT_INST_DIR' noPath  yes
 reportEnv '$WM_PROJECT_USER_DIR' noPath  no
 reportEnv '$WM_THIRD_PARTY_DIR'  noPath  yes
 hline
@@ -415,11 +412,12 @@ hline
 reportEnv '$FOAM_LIBBIN'      '$LD_LIBRARY_PATH'  yes
 reportEnv '$FOAM_SITE_LIBBIN' '$LD_LIBRARY_PATH'  no
 reportEnv '$FOAM_USER_LIBBIN' '$LD_LIBRARY_PATH'  no
+reportEnv '$FOAM_EXT_LIBBIN'  '$LD_LIBRARY_PATH'  maybe
 reportEnv '$MPI_ARCH_PATH'    '$LD_LIBRARY_PATH'  yes
 hline
 
 #------------------------------------------------------------------------------
-heading "Third-party software"
+heading "Software Components"
 hline
 echo "$(fixlen Software 9) $(fixlen Version 10) $(fixlen Location 10)"
 hline

bin/foamLog

@@ -28,11 +28,16 @@
 # Description
 #     Extract data for each time-step from a log file for graphing.
 #
+# Environment
+#     WM_PROJECT_API
+#     WM_PROJECT_DIR
+#     WM_PROJECT_SITE
+#
 #------------------------------------------------------------------------------
-Script=${0##*/}
-toolsDir=${0%/*}/tools
-siteDir="${WM_PROJECT_SITE:-${WM_PROJECT_DIR:-<unknown>}/site}"
-userDir=$HOME/.OpenFOAM
+Script="${0##*/}"
+toolsDir="${0%/*}/tools"
+groupDir="${WM_PROJECT_SITE:-${WM_PROJECT_DIR:-<unknown>}/site}"
+userDir="$HOME/.OpenFOAM"
 
 usage() {
     exec 1>&2
@@ -85,11 +90,11 @@ cat <<HELP
 
     The database ($Script.db) will taken from these locations:
         .
-        $userDir/$WM_PROJECT_VERSION
-        $userDir
-        $siteDir/$WM_PROJECT_VERSION
-        $siteDir
-        $WM_PROJECT_DIR/etc
+        $userDir/$WM_PROJECT_API/
+        $userDir/
+        $groupDir/$WM_PROJECT_API/etc/
+        $groupDir/etc/
+        $WM_PROJECT_DIR/etc/
         $toolsDir
 
     option -quiet : suppresses the default information and only prints the

bin/foamNewCase

@@ -4,7 +4,7 @@
 # \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
 #  \\    /   O peration     |
 #   \\  /    A nd           | Copyright (C) 2011 OpenFOAM Foundation
-#    \\/     M anipulation  |
+#    \\/     M anipulation  | Copyright (C) 2018 OpenCFD Ltd.
 #-------------------------------------------------------------------------------
 # License
 #     This file is part of OpenFOAM.
@@ -29,10 +29,15 @@
 #     Create a new case from a template for particular applications
 #     - requires rsync
 #
+# Environment
+#     WM_PROJECT_API
+#     WM_PROJECT_DIR
+#     WM_PROJECT_SITE
+#
 #------------------------------------------------------------------------------
-siteDir="${WM_PROJECT_SITE:-${WM_PROJECT_DIR:-<unknown>}/site}"
+groupDir="${WM_PROJECT_SITE:-${WM_PROJECT_DIR:-<unknown>}/site}"
 userDir="$HOME/.OpenFOAM"
-version="${WM_PROJECT_VERSION:-unknown}"
+projectApi="${WM_PROJECT_API:-unknown}"
 
 templateDir="appTemplates"
 
@@ -44,20 +49,22 @@ usage() {
 
 Usage: ${0##*/} [OPTION]
 options:
-  -app <name>       specify the application to use
-  -case <dir>       specify alternative case directory, default is the cwd
+  -app NAME         specify the application to use
+  -case DIR         specify alternative case directory, default is the cwd
   -list             list the applications available
-  -version <ver>    specify an alternative version (default: '$WM_PROJECT_VERSION')