... | ... | @@ -65,6 +65,73 @@ This is a very broad ranging topic, but here are some general ideas... |
|
|
const scalarField mergedWeights(globalFaces().gather(weights));
|
|
|
```
|
|
|
|
|
|
|
|
|
### Broadcast
|
|
|
|
|
|
_Sending data from the master rank to sub-ranks_
|
|
|
|
|
|
Prior to OpenFOAM-v2206, the `Foam::scatter()` function was used to
|
|
|
send data from the master rank to all sub-ranks. This function (which
|
|
|
used manual tree communication) is superseded by
|
|
|
`Pstream::broadcast()` which is based instead on MPI Bcast.
|
|
|
|
|
|
The OpenFOAM `Pstream::broadcast()` function corresponds an MPI Bcast,
|
|
|
but with serialize/deserialize wrappers (where needed).
|
|
|
For contiguous data types, it maps _exactly_ to an `MPI_Bcast`.
|
|
|
It is an no-op in non-parallel code.
|
|
|
|
|
|
When data are serialized/deserialized, it can be useful to bundle
|
|
|
several smaller data items into a single broadcast with the
|
|
|
`Pstream::broadcasts()` function.
|
|
|
|
|
|
|
|
|
### Plain Reductions
|
|
|
|
|
|
The OpenFOAM `Foam::reduce()` function corresponds to an MPI Allreduce,
|
|
|
but with serialize/deserialize wrappers (where needed). For primitive
|
|
|
data types (eg, bool, int, double, ...) it corresponds _exactly_ to
|
|
|
an `MPI_Allreduce`. It is an no-op in non-parallel code.
|
|
|
|
|
|
The `Foam::returnReduce` function is used in places where a reduced
|
|
|
value is required (eg, for an `if` statement), or where the value to
|
|
|
be reduced in read-only. For example,
|
|
|
```
|
|
|
Info<< "Total " << returnReduce(list.size(), sumOp<label>()) << nl;
|
|
|
```
|
|
|
|
|
|
The reduction of boolean values is treated as a special case, since
|
|
|
these be mapped directly to MPI intrinsic calls. Additionally, for the
|
|
|
return-reduce case, it can be extremely handy to use the implicit
|
|
|
bool conversion. For example,
|
|
|
```
|
|
|
// Has values everywhere?
|
|
|
if (returnReduceAnd(list.size())) ...
|
|
|
|
|
|
// Any values anywhere?
|
|
|
while (returnReduceOr(list.size()) && iter < 10) ...
|
|
|
```
|
|
|
|
|
|
The plain (non-returning versions) can be used as expected:
|
|
|
```
|
|
|
Pstream::reduceAnd(haveLib);
|
|
|
Pstream::reduceOr(hasError);
|
|
|
```
|
|
|
|
|
|
|
|
|
### Other Reductions
|
|
|
|
|
|
In addition to _regular_ reductions, it is also possible to perform
|
|
|
reduction with different types of containers and also with a combine
|
|
|
operation. The following are some of the common operations:
|
|
|
|
|
|
| Function | Note |
|
|
|
|---------------------|-------------------------------|
|
|
|
| combineReduce | combineGather + broadcast |
|
|
|
| listCombineReduce | listCombineGather + broadcast |
|
|
|
| mapCombineReduce | mapCombineGather + broadcast |
|
|
|
| allGatherList | gatherList + scatterList |
|
|
|
|
|
|
|
|
|
----
|
|
|
|
|
|
Copyright (C) 2022 OpenCFD Ltd.
|
... | ... | |