- Tested by writing volFields with the icoFoam's cavity test case
(boundary file is still needed in polyMesh)

- ASCII field files have some formatting issues (writing is done with
dictionary::write)

- OFCstream now inherits from OCharStream; rename foam-extend flavored
member functions to OF equivalents

- New regIOobject virtual member function "writeToStream" allows to
construct OFCstream within GeometricField

- The heavy data is now given to an OFCstream in the Field class, not UList.
Doing this makes it simple to ensure that every rank will call
OFCstream::write for every field.

- return instead of populating via parameter

- reduced overhead and bookkeeping
- non-blocking communication of slices and shared points
- use globalIndex for all internal accounting

- extracts more relevant information during construct or reset
  to allow inexpensive queries such as the number of
  internal/boundary faces and (non-processor) patch sizes etc.

  By consolidating more bookkeeping and functionality can
  greatly reduce the number of scans of input data performed.

- avoid fragile instance counter in favour of construct with index

- support two-stage construction, hide some functions, remove unused
  variables

- align naming more with OpenFOAM conventions.

- include sanity checks on boundary reading, PtrList management

- ensure that "boundary" information does not have more patches
  than known from the coherent format

- polyPatchList instead of List of raw pointers for additional safety.
  Avoids memory leaks, even if the caller doesn't use the data.

Gregor Weiss authored 1 year ago and Mark OLESEN committed 1 year ago

- integration notes (Mark Olesen):

  * left the MUST_READ/READ_IF_PRESENT flags untouched in the
    initializer to reduce noise of the commit (to be addressed)

- also generates a "polyMesh/coherent" dictionary with some boundary
  information etc. Can be useful for later detection of coherent vs
  non-coherent formats etc.

General
-------

- avoid parameter copying, more stringent test on bad input

- support writing primitives with globalIndex reference and labelPair

- obtain underlying component type and number from VectorSpace members
  instead of assuming that these are only 'scalar' and that the data
  buffer is non-null!

- add Reader(), Writer() factory methods for creating streams.
  * Simpler code and few header dependencies.

- style: method name sync_buffer() instead of bufferSync()

- support SYNC mode when reading/writing slice contents.
  Avoids using deferred and then sync on the entire buffer.
  Use get(), get_sync() methods instead of readToContainer helper

Repo
----

- make SliceStreamRepo singleton a std::unique_ptr.
  (avoids leakage on closure).

- add closeInstance() and endInstance() for extra management
  of the singleton

ENH: robuster handling of ADIOS config file

- use "system/adios-config.xml" or "system/adios-config.yaml"
  (was "system/config.xml") to make its intended purpose more transparent.

- although ADIOS does not require a config file, it will fail if the
  config file is specified and missing. Now check the availability of
  the config file(s) and unset if missing.

  NB: checks use the absolute global case-path for additional safety.

ENH: split off read buffer creation and size querying

- add missing totalSize() method

- replace the size() method, which was actually the local size
  with a range() method for querying the local dimensions

  range()     : queries the first component of Start() and Count()
  totalSize() : queries the first component of the Shape()

- add clear(), reset(), clone() methods to InitFromADIOS to simplify
  reuse and multiple queries etc.

ENH: add version (api) and sizes (label,scalar) as attributes

ENH: handle writing of multi-component types directly from SliceStream

- the sliceWritePrimitives helper routines should now be redundant

- cleaner integration of globalIndex:
  * support extract to globalIndex
  * use globalIndex at the components level (for gradual phase-out)

- global end-offsets are directly calculated (using MPI Allgather)

- global start-offsets are calculated with globalIndex::calcOffsets,
  without an intermediate list

- provide constant() DataComponent functor

  * simply returns the constant value with which it was constructed.
    Replace special-purpose start_from_myProcNo, count_two functions
    with equivalent constant() functors

- pass and return strings by reference

ENH: use factory forwarding for InitStrategies to simplify code

- std::unique_ptr ownership passed by move reference

STYLE: place DataComponent functions into OffsetStrategies namespace

- reduces clutter of the Foam namespace
- document their functionality

- replace internal labelPair typedef (std::pair<label,label>, not
  Foam::labelPair) with range_type to avoid possible confusion

- use internal typedefs for OffsetStrategy and DataComponent::base_ptr
  to avoid additional clutter in Foam namespace.

- use class instead of struct with private/protected content

- use OpenFOAM containers and naming conventions.

INT: add transitional methods to Offsets

- Offsets is slated for future removal since it is largely identically
  to what globalIndex provides, but provide compilation path for now

STYLE: simplify/extend sliceMeshHelper

- a namespace with some static functions and/or algorithms for
  handling some of the coherent mesh format details.

 * basic extraction for slice faces / points

 * categorization of upper-connected face neighbours

 * serialize/deserialize owners/faces :
   somewhat similar to CompactListList pack/unpack but
   templated on parameters to support std::vector etc.

  - partition-offsets  (was partitionStarts)
  - owner-offsets      (was ownerStarts)
  - neighbour          (was neighbours)
  - face-offsets       (was faceStarts)
  - face-points        (was faces)

STYLE: adjust class naming to reflect (current/future) purpose

- coherentMeshPolyMapper (old: FragmentPermutation)
- coherentProcessorPatch (old: ProcessorPatch)
- polyMeshCoherentMapper (old: SlicePermutation)

STYLE: use contains() for slice maps

- combine DataComponentFree into DataComponent

- make sliceMap header-only (will be removed)

GIT: remove remnant OffsetDecorator header

- this will presumably change again in the future

- ensure that operator<< and operator>> behave symmetrically

improve fileHandler support for redistributePar

See merge request Development/openfoam!605

mattijs authored 1 year ago and Andrew Heather committed 1 year ago

- uses read&write handlers
- fixes initialisation of partial handlers
- 'uniform' copied to wrong place
- -overwrite not deleting old processors dirs

mattijs authored 1 year ago and Andrew Heather committed 1 year ago

BUG: collated: cooperate with non-parallel

mattijs authored 2 years ago and Andrew Heather committed 1 year ago

- use file handlers to manage where meshes/fields are to be read and
  written.

mattijs authored 2 years ago and Andrew Heather committed 1 year ago

- code is compiled dynamically on the master node.
  In the normal (non-distributed) case, simply poll the NFS
  to see when it appears on the sub-procs.

  For a case with distributed roots, first broadcast it (via MPI)
  to the IO master nodes and then poll afterwards.

- on startup, detect and create missing processorXXX/ subdirectories
  on distributed filesystems

mattijs authored 2 years ago and Andrew Heather committed 1 year ago

- this delay the communication of file watches, which helps avoid
  communication deadlocks with master-only reading.

Co-authored-by: Mark Olesen <>

mattijs authored 1 year ago and Andrew Heather committed 1 year ago

- when reading, detect all clouds on all processors and uses this when
  reading fields. Similarly, when writing it uses writeOnProc to skip
  clouds that are empty on any particular processor.

Co-authored-by: Mark Olesen <>