Mark OLESEN authored 1 week ago and Mattijs Janssens committed 6 days ago

- avoids multiple definitions for various template specializations

Mark OLESEN authored 2 weeks ago and Mattijs Janssens committed 6 days ago

- can be used to pre-allocate space for internal+boundary field for
  flat addressing (issue #3364).

  For example:

      DynamicField<Type> fld
      (
          std::pair<label,label>
          (
              GeoMesh::size(mesh),
              GeoMesh::size(mesh)
            + (extra ? GeoMesh::boundary_size(mesh) : label(0))
          )
      );

ENH: base changes for GeometricField flags

- FieldBase 'localBoundaryConsistency_' and 'localBoundaryTolerance_'
  for central handling of local boundary consistency checks.

- FieldBase 'unifiedGeometricField' for future extensions to GeometricField

- allocate communicator with only those ranks that have
patch faces
- use this communicator everywhere
- communicator preserved during mesh motion

- split/duplicate functionality

- rework inter-node/intra-node handling to allow selection of
  splitting based on 'shared' or hostname (default).

- always creates node communicators at startup:
  * commInterNode() - between nodes
  * commLocalNode() - within a node

- world-comm is now always a duplicate of MPI_COMM_WORLD to provide
  better separation from other processes.

NB:
   the inter-node comm is a slight exception to other communicators
   in that we always retain its list of (global) ranks, even if the
   local process is not in that communicator.
   This can help when constructing topology-aware patterns.

FIX: non-participating ranks still had knowledge of their potential siblings

- after create by group, the procIDs_ of non-participating ranks
  should be empty (except for the inter-node exception)

- the maxCommsSize variable is used to 'chunk' large data transfers
  (eg, with PstreamBuffers) into a multi-pass send/recv sequence.

  The send/recv windows for chunk-wise transfers:

      iter    data window
      ----    -----------
      0       [0, 1*chunk]
      1       [1*chunk, 2*chunk]
      2       [2*chunk, 3*chunk]
      ...

  Since we mostly send/recv in bytes, the current internal limit
  for MPI counts (INT_MAX) can be hit rather quickly.

  The chunking limit should thus also be INT_MAX, but since it is
  rather tedious to specify such large numbers, can instead use

      maxCommsSize = -1

  to specify (INT_MAX-1) as the limit.
  The default value of maxCommsSize = 0 (ie, no chunking).

Note
~~~~
  In previous versions, the number of chunks was determined by the
  sender sizes. This required an additional MPI_Allreduce to establish
  an overall consistent number of chunks to walk. This additional
  overhead each time meant that maxCommsSize was rarely actually
  enabled.

  We can, however, instead rely on the send/recv buffers having been
  consistently sized and simply walk through the local send/recvs until
  no further chunks need to be exchanged. As an additional enhancement,
  the message tags are connected to chunking iteration, which allows
  the setup of all send/recvs without an intermediate Allwait.

ENH: extend UPstream::probeMessage to use int64 instead of int for sizes

- "buffered" corresponds to MPI_Bsend (buffered send),
  whereas the old name "blocking" is misleading since the
  regular MPI_Send also blocks until completion
  (ie, buffer can be reused).

ENH: IPstream::read() returns std::streamsize instead of label (#3152)

- previously returned a 'label' but std::streamsize is consistent with
  the input parameter and will help with later adjustments.

- use <label> instead of <int> for internal accounting of the message
  size, for consistency with the underyling List<char> buffers used.

- improve handling for corner case of IPstream receive with
  non-blocking, although this combination is not used anywhere

- consistent with etc/controlDict default

STYLE: update banner message for collated

- was previously limited to 'char' whereas gatherv/scatterv
  already supported various integer and float types

STYLE: rebundle allToAll declarations with macros

ENH: provide a version of allToAllConsensus returning the Map

- simplifies use and avoids ambiguities in the send/recv parameters

- the Map version will now also transmit zero value data if they exist
  in the Map. Unlike the List version, zero values are not necessary to
  signal connectivity with a Map.

COMP: forwarding template parameters for NBX routines

ENH: consolidate PstreamBuffers size exchange options

- had a variety of nearly identical backends for all-to-all,
  gather/scatter. Now combined internally with a dispatch enumeration
  which provides better control over which size exchange algorithm
  is used.

DEFEATURE: remove experimental full-NBX PstreamBuffers variant

- no advantages seen compared to the hybrid NBX/PEX approach.
  Removal reduces some code cruft.

DEFE...

- to prevent NBX from intercepting previous messages an initial
  barrier can help with synchronization. (Gregor Weiss)

- previously had an additional stack for freedRequests_,
  which were used to 'remember' locations into the list of
  outstandingRequests_ that were handled by 'waitRequest()'.

  This was principally done for sanity checks on shutdown,
  but we now just test for any outstanding requests that
  are *not* MPI_REQUEST_NULL instead (much simpler).

  The framework with freedRequests_ also had a provision to 'recycle'
  them by popping from that stack, but this is rather fragile since it
  would only triggered by some collectives
  (MPI_Iallreduce, MPI_Ialltoall, MPI_Igather, MPI_Iscatter)
  with no guarantee that these will all be properly removed again.
  There was also no pruning of extraneous indices.

ENH: consolidate internal reset/push of requests

- replace duplicate code with inline functions
  reset_request(), push_request()

ENH: null out trailing requests

- extra safety (paranoia) for the UPstream::Request versions
  of finishedRequests(), waitAnyRequest()

CONFIG: document nPollProcInterfaces in etc/controlDict

- still experimental, but at least make the keyword known

- can be beneficial with increasing number of ranks

- simplifies code, consistent with other matrix transfer functions.
  Use a setter method.

STYLE: AMIInterpolation::upToDate(bool) setter method

ENH: add guards to avoid float-compressed transfer of integral types

STYLE: drop unused debug member from abstract interface classes

- since ensight format is always float and also always written
  component-wise, perform the double -> float narrowing when
  extracting the components.  This reduces the amount of data
  transferred between processors.

ENH: avoid vtk/ensight parallel communication of empty messages

- since ensight writes by element type (eg, tet, hex, polyhedral) the
  individual written field sections will tend to be relatively sparse.
  Skip zero-size messages, which should help reduce some of the
  synchronization bottlenecks.

ENH: use 'data chunking' when writing ensight files in parallel

- since ensight fields are written on a per-element basis, the
  corresponding segment can become rather sparsely distributed. With
  'data chunking', we attempt to get as many send/recv messages in
  before flushing the buffer for writing. This should make the
  sequential send/recv less affected by the IO time.

ENH: allow use of an external buffer when writing ensight components

STYLE: remove last vestiges of autoPtr<ensightFile> for output routines

- update annotated dicts, remove tabs

- use point1/point2 for cylinder sources

- the very old 'writer' class was fully stateless and always templated
  on an particular output type.

  This is now replaced with a 'coordSetWriter' with similar concepts
  as previously introduced for surface writers (#1206).

  - writers change from being a generic state-less set of routines to
    more properly conforming to the normal notion of a writer.

  - Parallel data is done *outside* of the writers, since they are used
    in a wide variety of contexts and the caller is currently still in
    a better position for deciding how to combine parallel data.

ENH: update sampleSets to sample on per-field basis (#2347)

- sample/write a field in a single step.

- support for 'sampleOnExecute' to obtain values at execution
  intervals without writing.

- support 'sets' input as a dictionary entry (as well as a list),
  which is similar to the changes for sampled-surface and permits use
  of changeDictionary to modify content.

- globalIndex for gather to reduce parallel communication, less code

- qualify the sampleSet results (properties) with the name of the set.
  The sample results were previously without a qualifier, which meant
  that only the last property value was actually saved (previous ones
  overwritten).

  For example,
  ```
    sample1
    {
        scalar
        {
            average(line,T) 349.96521;
            min(line,T)     349.9544281;
            max(line,T)     350;
            average(cells,T) 349.9854619;
            min(cells,T)    349.6589286;
            max(cells,T)    350.4967271;
            average(line,epsilon) 0.04947733869;
            min(line,epsilon) 0.04449639927;
            max(line,epsilon) 0.06452856475;
        }
        label
        {
            size(line,T)    79;
            size(cells,T)   1720;
            size(line,epsilon) 79;
        }
    }
  ```

ENH: update particleTracks application

- use globalIndex to manage original parcel addressing and
  for gathering. Simplify code by introducing a helper class,
  storing intermediate fields in hash tables instead of
  separate lists.

ADDITIONAL NOTES:

- the regionSizeDistribution largely retains separate writers since
  the utility of placing sum/dev/count for all fields into a single file
  is questionable.

- the streamline writing remains a "soft" upgrade, which means that
  scalar and vector fields are still collected a priori and not
  on-the-fly.  This is due to how the streamline infrastructure is
  currently handled (should be upgraded in the future).

Minor clean-up

- use common base for handling scheme lookups.
  Extended to support local injection of schemes into the dictionaries
  during runtime.

- if the system/decomposeParDict is missing, skip check for matching
  number of processor dirs. This can make job dispatch easier.
  Does not apply if -decomposeParDict was explicitly specified.

STYLE: adjust naming of host/slaves in argList

- location and naming more consistent with other controls

    old: DebugSwitch  'level'
    new: InfoSwitch   'outputLevel'

- previously always called dlclose on opened libraries when destroying
  the dlLibraryTable. However, by force closing the libraries the
  situation can arise that the library is missing its own code that it
  needs on unload (#1524). This is also sometimes evident when closing
  VTK libraries for runTimePostProcessing (#354, #1585).

- The new default is to not forcibly dlclose any libraries, unless
  the dlcloseOnTerminate OptimisationSwitch specifies otherwise.

  - The dlLibraryTable::close() method can be used to explicitly close
    all libraries and clear the list.

  - The dlLibraryTable::clear() method now only clears the entries,
    without a dlclose.

- better distinction between types of algorithms.
  Easier for future deprecation/replacement.

- timeVaryingUniformFixedValue -> uniformFixedValue

- allows a variety of functions (eg, coded, expressions, tables, ...)

- more similarity to finiteVolume patch type

STYLE: remove unused timeVarying... from etc/controlDict

This change abstracts out the reading of "boundaryData". It should
now support OpenFOAM headers and with that also binary input.

- remove (unused) Istream constructors, prune some unused methods,
  rationalize write() vs writeDict().
  Deprecate inconsistent construction order.

- handle empty names for ".ftr" surface patches (for plain triSurface
  format) with double-quoted strings for more reliable streaming.
  Written on a single line.

  This is _backward_ compatible, but if users have been parsing these
  files manually, they will need to adjust their code.

Previously:
```
  (
  frt-fairing:001%1
  empty

  windshield:002%2
  empty
  ...
  )
```

Updated (with example handling of empty name):
```
  (
  frt-fairing:001%1 empty

  windshield:002%2 ""
  ...
  )
```

- now use debug 2 for scanner and debug 4 for parser.
  Provided better feedback about what is being parsed (debug mode)

- relocate debug application to applications/tools/foamExprParserInfo

- this cannot be left as a configurable value (on windows), since it
  needs to be enabled even prior to reading the etc/controlDict file,
  in case the OpenFOAM installation path itself contains spaces.