- this compact form shows the subscription per host in the unsorted
  mpi order

      nProcs : 18
      Hosts  :
      (
          (node1 6)
          (node2 8)
          (node3 4)
      )

  This provides a succinct overview of which hosts have been
  subscribed or oversubscribed.

- The longer list of "slave.pid" ... remains available on the
  InfoSwitch 'writeHosts'

- can inadvertently arise from TAB-completion, but an option without a
  name doesn't make sense, we trap this and emit a warning.

- With many processors, the number of entries becomes quite large.

  New controlDict InfoSwitches: "writeSlaves", "writeRoots".

- make some IOobject writing routines non-templated.

- relocate more IOobject methods into the IOobjectI.H
  (generally cleanup the main header).

- end user can easily see which arch (endian, label/scalar size) was
  used from the log output, or via -help usage

ENH: noiseModels - updated parallel usage (no longer needs proc dirs) and apply input data validation

Related to #482 but this one is the handling inside argList.

Henry Weller authored Aug 03, 2017 and Andrew Heather committed Aug 03, 2017

Patch contributed by Timo Niemi, VTT.
Resolves bug-report https://bugs.openfoam.org/view.php?id=2645

Original commit message:
------------------------

Parallel IO: New collated file format

When an OpenFOAM simulation runs in parallel, the data for decomposed fields and
mesh(es) has historically been stored in multiple files within separate
directories for each processor.  Processor directories are named 'processorN',
where N is the processor number.

This commit introduces an alternative "collated" file format where the data for
each decomposed field (and mesh) is collated into a single file, which is
written and read on the master processor.  The files are stored in a single
directory named 'processors'.

The new format produces significantly fewer files - one per field, instead of N
per field.  For large parallel cases, this avoids the restriction on the number
of open files imposed by the operating system limits.

The file writing can be threaded allowing the simulation to continue running
while the data is being written to file.  NFS (Network File System) is not
needed when using the the collated format and additionally, there is an option
to run without NFS with the original uncollated approach, known as
"masterUncollated".

The controls for the file handling are in the OptimisationSwitches of
etc/controlDict:

OptimisationSwitches
{
    ...

    //- Parallel IO file handler
    //  uncollated (default), collated or masterUncollated
    fileHandler uncollated;

    //- collated: thread buffer size for queued file writes.
    //  If set to 0 or not sufficient for the file size threading is not used.
    //  Default: 2e9
    maxThreadFileBufferSize 2e9;

    //- masterUncollated: non-blocking buffer size.
    //  If the file exceeds this buffer size scheduled transfer is used.
    //  Default: 2e9
    maxMasterFileBufferSize 2e9;
}

When using the collated file handling, memory is allocated for the data in the
thread.  maxThreadFileBufferSize sets the maximum size of memory in bytes that
is allocated.  If the data exceeds this size, the write does not use threading.

When using the masterUncollated file handling, non-blocking MPI communication
requires a sufficiently large memory buffer on the master node.
maxMasterFileBufferSize sets the maximum size in bytes of the buffer.  If the
data exceeds this size, the system uses scheduled communication.

The installation defaults for the fileHandler choice, maxThreadFileBufferSize
and maxMasterFileBufferSize (set in etc/controlDict) can be over-ridden within
the case controlDict file, like other parameters.  Additionally the fileHandler
can be set by:
- the "-fileHandler" command line argument;
- a FOAM_FILEHANDLER environment variable.

A foamFormatConvert utility allows users to convert files between the collated
and uncollated formats, e.g.
    mpirun -np 2 foamFormatConvert -parallel -fileHandler uncollated

An example case demonstrating the file handling methods is provided in:
$FOAM_TUTORIALS/IO/fileHandling

The work was undertaken by Mattijs Janssens, in collaboration with Henry Weller.

Henry Weller authored Jun 11, 2017 and Andrew Heather committed Jun 11, 2017

Initially the listSwitches functions depended directly on argList functionality
but this has now been factored out so that the listSwitches functions are more
general and require only debug functionality.

Henry Weller authored Jun 10, 2017 and Andrew Heather committed Jun 10, 2017

Provides better context for the available boundary conditions, fvOptions,
functionObjects etc. and thus returns only those available to and compatible
with the particular application.

e.g.

pimpleFoam -help

Usage: pimpleFoam [OPTIONS]
options:
  -case <dir>       specify alternate case directory, default is the cwd
  -listFunctionObjects
                    List functionObjects
  -listFvOptions    List fvOptions
  -listRegisteredSwitches
                    List switches registered for run-time modification
  -listScalarBCs    List scalar field boundary conditions (fvPatchField<scalar>)
  -listSwitches     List switches declared in libraries but not set in
                    etc/controlDict
  -listTurbulenceModels
                    List turbulenceModels
  -listUnsetSwitches
                    List switches declared in libraries but not set in
                    etc/controlDict
  -listVectorBCs    List vector field boundary conditions (fvPatchField<vector>)
  -noFunctionObjects
                    do not execute functionObjects
  -parallel         run in parallel
  -postProcess      Execute functionObjects only
  -roots <(dir1 .. dirN)>
                    slave root directories for distributed running
  -srcDoc           display source code in browser
  -doc              display application documentation in browser
  -help             print the usage

pimpleFoam listTurbulenceModels

pimpleFoam -listTurbulenceModels
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  dev                                   |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : dev-39c46019e44f
Exec   : pimpleFoam -listTurbulenceModels
Date   : Jun 10 2017
Time   : 21:37:49
Host   : "dm"
PID    : 675
Case   : /home/dm2/henry/OpenFOAM/OpenFOAM-dev
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
SetNaN : Initialising allocated memory to NaN (FOAM_SETNAN).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Turbulence models
3
(
LES
RAS
laminar
)

RAS models
18
(
LRR
LamBremhorstKE
LaunderSharmaKE
LienCubicKE
LienLeschziner
RNGkEpsilon
SSG
ShihQuadraticKE
SpalartAllmaras
kEpsilon
kOmega
kOmegaSST
kOmegaSSTLM
kOmegaSSTSAS
kkLOmega
qZeta
realizableKE
v2f
)

LES models
10
(
DeardorffDiffStress
Smagorinsky
SpalartAllmarasDDES
SpalartAllmarasDES
SpalartAllmarasIDDES
WALE
dynamicKEqn
dynamicLagrangian
kEqn
kOmegaSSTDES
)

Further work will be needed to support the -listTurbulenceModels option in
multiphase solvers.

- use InfoSwitch to disable, or via static method.

- respect the state of the argList banner when deciding to emit
  initialization information. Can otherwise end up with unwanted
  output rubbish on things like foamDictionary and foamListTimes.

Fixes #420.

- this implies that jobControl is a user-resource for OpenFOAM.
  It was previously located under $WM_PROJECT_INST_DIR/jobControl,
  but few users will have write access there.

- an unset FOAM_JOB_DIR variable is treated as "~/.OpenFOAM/jobControl",
  which can partially reduce environment clutter.

- provide argList::noJobInfo() to conveniently suppress job-info on an
  individual basis for short-running utilities (eg, foamListTimes) to
  avoid unneeded clutter.

- can be used directly from within an application or function-object.
  Makes it available for dictionaries.

- adjust so that the final -option is taken. This is consistent with
  other Unix tools.

ENH: short-circuit to avoid directory check on non-master node

fileModificationSkew: now a floating-point number to support sub-second
specification.

Patch contributed by Mattijs Janssens

- Cleanup/centralize handling of -decomposeParDict by relocating
  common code into argList. Ensures that all processes receive
  identical information about the -decomposeParDict opton.

- Only use alternative decomposeParDict for simpleFoam/motorBike
  tutorial so that this will be included in the test loop for snappy.

- Added Mattijs' fix for surfaceRedistributePar.

The normal library system() command uses 'fork', which causes
problems on IB+OPENMPI.

STYLE: add Foam:: qualifier to system calls to make them easier to spot.

splitMeshRegions: handle flipping of faces for surface fields

subsetMesh: subset dimensionedFields

decomposePar: use run-time selection of decomposition constraints. Used to
    keep cells on particular processors. See the decomposeParDict in

$FOAM_UTILITIES/parallel/decomposePar:
  - preserveBaffles: keep baffle faces on same processor
  - preserveFaceZones: keep faceZones owner and neighbour on same processor
  - preservePatches: keep owner and neighbour on same processor. Note: not
    suitable for cyclicAMI since these are not coupled on the patch level
  - singleProcessorFaceSets: keep complete faceSet on a single processor
  - refinementHistory: keep cells originating from a single cell on the
    same processor.

decomposePar: clean up decomposition of refinement data from snappyHexMesh

reconstructPar: reconstruct refinement data (refineHexMesh, snappyHexMesh)

reconstructParMesh: reconstruct refinement data (refineHexMesh, snappyHexMesh)

redistributePar:
  - corrected mapping surfaceFields
  - adding processor patches in order consistent with decomposePar

argList: check that slaves are running same version as master

fvMeshSubset: move to dynamicMesh library

fvMeshDistribute:
  - support for mapping dimensionedFields
  - corrected mapping of surfaceFields

parallel routines: allow parallel running on single processor

Field: support for
  - distributed mapping
  - mapping with flipping

mapDistribute: support for flipping

AMIInterpolation: avoid constructing localPoints

    Executes application functionObjects to post-process existing results.

    If the "dict" argument is specified the functionObjectList is constructed
    from that dictionary otherwise the functionObjectList is constructed from
    the "functions" sub-dictionary of "system/controlDict"

    Multiple time-steps may be processed and the standard utility time
    controls are provided.

This functionality is equivalent to execFlowFunctionObjects but in a
more efficient and general manner and will be included in all the
OpenFOAM solvers if it proves effective and maintainable.

The command-line options available with the "-postProcess" option may be
obtained by

simpleFoam -help -postProcess

Usage: simpleFoam [OPTIONS]
options:
  -case <dir>       specify alternate case directory, default is the cwd
  -constant         include the 'constant/' dir in the times list
  -dict <file>      read control dictionary from specified location
  -latestTime       select the latest time
  -newTimes         select the new times
  -noFunctionObjects
                    do not execute functionObjects
  -noZero           exclude the '0/' dir from the times list, has precedence
                    over the -withZero option
  -parallel         run in parallel
  -postProcess      Execute functionObjects only
  -region <name>    specify alternative mesh region
  -roots <(dir1 .. dirN)>
                    slave root directories for distributed running
  -time <ranges>    comma-separated time ranges - eg, ':10,20,40:70,1000:'
  -srcDoc           display source code in browser
  -doc              display application documentation in browser
  -help             print the usage

Henry G. Weller
CFD Direct Ltd.

The signal handling can now be set and unset. This latter is important
for external libraries (e.g. VTK/OpenGL) that do not work with it.

The signal handling can now be set and unset. This latter is important
for external libraries (e.g. VTK/OpenGL) that do not work with it.

- redistributePar to have almost (complete) functionality of decomposePar+reconstructPar
- low-level distributed Field mapping
- support for mapping surfaceFields (including flipping faces)
- support for decomposing/reconstructing refinement data

- redistributePar to have almost (complete) functionality of decomposePar+reconstructPar
- low-level distributed Field mapping
- support for mapping surfaceFields (including flipping faces)
- support for decomposing/reconstructing refinement data