- don't need to retain group references globally.
  Just retrieve parent group reference on-demand and discard
  immediately afterwards.

- with (nPollProcInterfaces < 0) it does the following:

  - loop, waiting for some requests to finish
  - for each out-of-date interface, check if its associated
    requests have now finished (ie, the ready() check).
  - if ready() -> call updateInterfaceMatrix()

  In contrast to (nPollProcInterfaces > 0) which loops a specified
  number of times with several calls to MPI_Test each time, the
  (nPollProcInterfaces < 0) variant relies on internal MPI looping
  within MPI_Waitsome to progress communication.

  The actual dispatch still remains non-deterministic (ie, waiting for
  some requests to finish does not mean that any particular interface
  is eligible for update, or in any particular order). However, using
  Waitsome places the tight looping into the MPI layer, which results
  in few calls and eliminates behaviour dependent on the value of
  nPollProcInterfaces.

TUT: add polling to windAroundBuildings case (for testing purposes)

- fewer calls, potentially more consistent

ENH: update sendRequest state after recvRequest wait

- previously had this type of code:

    // Treat send as finished when recv is done
    UPstream::waitRequest(recvRequest_);
    recvRequest_ = -1;
    sendRequest_ = -1;

  Now refined as follows:

    // Require receive data. Update the send request state.
    UPstream::waitRequest(recvRequest_);
    recvRequest_ = -1;
    if (UPstream::finishedRequest(sendRequest_)) sendRequest_ = -1;

  Can potentially investigate with requiring both,
  but this may be over-contrained.
  Example,

    // Require receive data, but also wait for sends too
    UPstream::waitRequestPair(recvRequest_, sendRequest_);

- checks requests from completion, returning true when some requests
  have completed and false when there are no active requests.
  This allows it to be used in a polling loop to progress MPI
  and then respond when as requests become satisfied.

  When using as part of a dispatch loop, waitSomeRequests() is
  probably more efficient than calling waitAnyRequest() and can help
  avoid biasing which client requests are serviced.

  Takes an optional return parameter, to retrieve the indices,
  but more importantly to avoid inner-loop reallocations.

  Example,

      DynamicList<int> indices;
      while (UPstream::waitSomeRequests(startRequest, &indices))
      {
          // Dispatch something ....
      }

      // Reset list of outstanding requests with 'Waitall' for safety
      UPstream::waitRequests(startRequest);

  ---

  If only dealing with single items and an index is required for
  dispatching, it can be better to use a list of UPstream::Request
  instead.

  Example,

      List<UPstream::Request> requests = ...;

      label index = -1;
      while ((index = UPstream::waitAnyRequest(requests)) >= 0)
      {
          // Do something at index
      }

ENH: pair-wise wrappers for MPI_Test or MPI_Wait

- for send/recv pairs of requests, can bundle both together and use a
  single MPI_Testsome and MPI_Waitall instead of two individual
  calls.

- previously had an additional stack for freedRequests_,
  which were used to 'remember' locations into the list of
  outstandingRequests_ that were handled by 'waitRequest()'.

  This was principally done for sanity checks on shutdown,
  but we now just test for any outstanding requests that
  are *not* MPI_REQUEST_NULL instead (much simpler).

  The framework with freedRequests_ also had a provision to 'recycle'
  them by popping from that stack, but this is rather fragile since it
  would only triggered by some collectives
  (MPI_Iallreduce, MPI_Ialltoall, MPI_Igather, MPI_Iscatter)
  with no guarantee that these will all be properly removed again.
  There was also no pruning of extraneous indices.

ENH: consolidate internal reset/push of requests

- replace duplicate code with inline functions
  reset_request(), push_request()

ENH: null out trailing requests

- extra safety (paranoia) for the UPstream::Request versions
  of finishedRequests(), waitAnyRequest()

CONFIG: document nPollProcInterfaces in etc/controlDict

- still experimental, but at least make the keyword known

- mechanism has been unused for at least a decade or more
  (or was never used). Message tags are assigned on an ad hoc basis
  locally when collision avoidance is necessary.

- not currently used, but it is possible that communicator allocation
  modifies the list of sub-ranks. Ensure that the correct size is used
  when (re)initialising the linear/tree structures.

STYLE: adjust MPI test applications

- remove some clutter and unneeded grouping.
  Some ideas for host-only communicators

- feature angle compared to real angle
- stop agglomerating if number of marked edges does not change

- allow reporting even when profiling is suspended

- consolidate reporting into profilingPstream itself
  (avoids code scatter).

Example of possible advanced use for timing only one section of
code:

    ====
    // Profile local operations
    profilingPstream::enable();

    ... do something

    // Don't profile elsewhere
    profilingPstream::suspend();
    ====

- separate broadcast times from reduce/gather/scatter time
- separate wait times from all-to-all time
- support invocation counts, split off requests time/count
  from others to avoid flooding the counts

- support 'detail' switch to increase the output information.
  Format may change in the future

STYLE: prefer GeometricField::New factory methods

- attributes such as assignable(), coupled() etc

- common patchField types: calculatedType(), zeroGradientType() etc.
  This simplifies reference to these types without actually needing a
  typed patchField version.

ENH: add some basic patchField types to fieldTypes namespace

- allows more general use of the names

ENH: set extrapolated/calculated from patchInternalField directly

- avoids intermediate tmp

- avoid dependency on etc/openfoam (location may change)

- with the current handling of small edges (finite-area), the LSQ
  vectors can result in singular/2D tensors. However, the regular
  2D handling in field inv() only detects based on the first element.

  Provide a 'failsafe' inv() method for symmTensor and tensor that
  follows a similar logic for avoiding zero determinates, but it is
  applied on a per element basis, instead of deciding based on the
  first field element.

  The symmTensor::inv(bool) and tensor::inv(bool) methods have a
  fairly modest additional overhead.

- unroll the field inv() function to avoid creating an intermediate
  field.  Reduce the number of operations when adjusting/re-adjusting
  the diagonal.

- for cases where a 3D tensor is being used to represent 2D content,
  the determinant is zero. Can use inv2D(excludeDirection) to compensate
  and invert as if it were only 2D.

ENH: consistent definitions for magSqr of symmTensors, diagSqr() norm

COMP: return scalar not component type for magSqr

- had inconsistent definitions with SymmTensor returning the component
  type and Tensor returning scalar. Only evident with complex.

- had an optimisation to skip attempted reading for zero-sized
  patches. But this leads to inconsistency in the code branching, thus
  removing it.

- when only a partial stacktrace is desirable.

ENH: add stack trace decorators

- the 0-th frame is always printStack(), so skip that and emit
  some headers/footers instead. Eg,

  [stack trace]
  =============
  #1  Foam::SymmTensor<double> Foam::inv<double>(...)
  #2  Foam::inv(Foam::UList<Foam::SymmTensor<double>> const&) ...
  ...
  =============

- data_bytes(), size_bytes() methods to support broadcasting or
  gather/scatter content. Additional construct from raw bytes
  to support transmitting content.

- can be easier to specify than copying into a dimensioned type.

  Eg,
      volSymmTensorField(io, mesh, Zero, dimLength);

      fluxFieldType
      (
          io
          mesh,
          Zero,
          Uf.dimensions()*dimArea/dimTime
      );

  vs
      volSymmTensorField
      (
          io,
          mesh,
          dimensionedSymmTensor("0", dimLength, Zero)
      );

      fluxFieldType
      (
          io
          mesh,
          dimensioned<typename flux<Type>::type>
          (
              Uf.dimensions()*dimArea/dimTime, Zero
          )
      );

- missed consistency in a few places.

- return nullptr (with automatic conversion to tmp) on failures
  instead of tmp<....>(nullptr), for cleaner coding.

INT: add support for an 'immovable' tmp pointer

- this idea is from openfoam.org, to allow creation of a tmp that is
  protected from having its memory reclaimed in field operations

ENH: tmp NewImmovable factory method, forwards as immovable/movable

mattijs authored 2 years ago and Mark OLESEN committed 2 years ago

- no-op implementations, but makes the call to
  GeometricBoundaryField::evaluate() less dependent on PatchField type

- add updated()/manipulatedMatrix() methods to faePatchField,
  fvsPatchField etc. These are mostly no-ops, but provide name
  compatible with fvPatchField etc.

- self-documenting

- similar to UPstream::parRun(), the setter returns the previous value.

  The accessors are prefixed with 'comm':
  Eg, commGlobal(), commWarn(), commWorld(), commSelf().
  This distinguishes them from any existing variables (eg, worldComm)
  and arguably more similar to MPI_COMM_WORLD etc...

  If demand-driven communicators are added in the future, the function
  call syntax can help encapsulate that.

  Previously:

      const label oldWarnComm = UPstream::warnComm;
      const label oldWorldComm = UPstream::worldComm;
      UPstream::warnComm = myComm;
      UPstream::worldComm = myComm;
      ...

      UPstream::warnComm = oldWarnComm;
      UPstream::worldComm = oldWorldComm;

  Now:
      const label oldWarnComm = UPstream::commWarn(myComm);
      const label oldWorldComm = UPstream::commWorld(myComm);
      ...

      UPstream::commWarn(oldWarnComm);
      UPstream::commWorld(oldWorldComm);

STYLE: check (warnComm >= 0) instead of (...

- deprecate timeVaryingUniformFixed (now redundant)

- base level now explicity equivalent to LAZY_READ with overriding as
  required

- clearer documentation for reading of "value" for faPatchField

- constructing with valueRequired as a bool is still supported,
  but now also support more refined requirements
  (eg, NO_READ, MUST_READ, LAZY_READ)

- continue with LAZY_READ for finite-area fields

STYLE: use readValueEntry and Field assign to simplify code

ENH: improved point-cell and cell-point topology methods (#2715)

See merge request Development/openfoam!597