Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • openfoam openfoam
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 426
    • Issues 426
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 9
    • Merge requests 9
  • Deployments
    • Deployments
    • Releases
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Development
  • openfoamopenfoam
  • Issues
  • #2179
Closed
Open
Issue created Aug 07, 2021 by Pablo Usyaopin@usyaop22

mapFieldsPar problem when running in Parallel

Summary

When executing the mapFieldsPar in parallel trying to map one box with 2 elements into another box of 2 or 4, or more elements I encounter a out of index error. When executing the same in serial it works fine.

Steps to reproduce

In the target source I execute the following:

mpirun -np 2 mapFieldsPar /decayIsoTurb4.1_copyWithLessTime -sourceTime latestTime -parallel

What is the current bug behaviour?

/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  v2006                                 |
|   \\  /    A nd           | Website:  www.openfoam.com                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 5e660c36e9-20201103 OPENFOAM=2006 patch=201012
Arch   : "LSB;label=32;scalar=64"
Exec   : mapFieldsPar /home/pablo/Desktop/ApplicationRuhr/Data/paperIlana/twoPointCorrEnric/decayIsoTurb4.1_copyWithLessTime -sourceTime latestTime -parallel
Date   : Aug 07 2021
Time   : 10:09:18
Host   : pablo-XPS-13-9370
PID    : 13775
I/O    : uncollated
Case   : /home/pablo/Desktop/ApplicationRuhr/Data/paperIlana/twoPointCorrEnric/decayIsoTurb4.1_trialWithLessElements
nProcs : 2
Hosts  :
(
    (pablo-XPS-13-9370 2)
)
Pstream initialized with:
    floatTransfer      : 0
    nProcsSimpleSum    : 0
    commsType          : nonBlocking
    polling iterations : 0
trapFpe: Floating point exception trapping enabled (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 5, maxFileModificationPolls 20)
allowSystemOperations : Allowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Source: "/home/pablo/Desktop/ApplicationRuhr/Data/paperIlana/twoPointCorrEnric" "decayIsoTurb4.1_copyWithLessTime"
Target: "/home/pablo/Desktop/ApplicationRuhr/Data/paperIlana/twoPointCorrEnric" "decayIsoTurb4.1_trialWithLessElements"

Create databases as time
I/O    : uncollated
Case   : /home/pablo/Desktop/ApplicationRuhr/Data/paperIlana/twoPointCorrEnric/decayIsoTurb4.1_copyWithLessTime
nProcs : 2
Hosts  :
(
    (pablo-XPS-13-9370 2)
)
Pstream initialized with:
    floatTransfer      : 0
    nProcsSimpleSum    : 0
    commsType          : nonBlocking
    polling iterations : 0
Overriding DebugSwitches according to controlDict
    meshToMesh      2;


Source time: 0
Target time: 0

Create meshes

Source mesh size: 2     Target mesh size: 2


Creating and mapping fields for time 0

Creating mesh-to-mesh addressing for region0 and region0 regions using cellVolumeWeight

    From Foam::label Foam::meshToMesh::calcDistribution(const Foam::polyMesh&, const Foam::polyMesh&) const
    in file meshToMesh/meshToMeshParallelOps.C at line 70 
    Meshes split across multiple processors
meshToMesh: Using AABBTree method

    From Foam::autoPtr<Foam::mapDistribute> Foam::meshToMesh::calcProcMap(const Foam::polyMesh&, const Foam::polyMesh&) const
    in file meshToMesh/meshToMeshParallelOps.C at line 176
    Determining extent of src mesh per processor:
        proc    bb
        0       2{(-0.0003 -0.0003 -0.0003) (0.0203 0.0203 0.0103)}
        1       2{(-0.0003 -0.0003 0.0097) (0.0203 0.0203 0.0203)}
[0] Of my 1 target cells I need to send to:
[0]     proc    cells
[0]     0       1
[0]     1       1
[1] Of my 1 target cells I need to send to:
[1]     proc    cells
[1]     0       1
[1]     1       1
[0] tgtProc:0 sending tgt cell 0[0] to srcProc 0
[1] tgtProc:1 sending tgt cell 0[1] to srcProc 0
[0] tgtProc:0 sending tgt cell 0[0] to srcProc 1
[1] tgtProc:1 sending tgt cell 0[1] to srcProc 1
[0] Target mesh send sizes[0]: points=8, faces=6, nInternalFaces=0, faceOwn=6, faceNbr=6, cellIDs=1
[0] Target mesh send sizes[1]: points=8, faces=6, nInternalFaces=0, faceOwn=6, faceNbr=6, cellIDs=1
[1] Target mesh send sizes[0]: points=8, faces=6, nInternalFaces=0, faceOwn=6, faceNbr=6, cellIDs=1
[1] Target mesh send sizes[1]: points=8, faces=6, nInternalFaces=0, faceOwn=6, faceNbr=6, cellIDs=1
[0] Additional internal face between procs:0 and 1 across local face 0
[0] Additional internal face between procs:0 and 1 across local face 0
[1] Additional internal face between procs:0 and 1 across local face 0
[1] Additional internal face between procs:0 and 1 across local face 0
[0] Additional internal face between procs:0 and 1 across local face 0
[1] Additional internal face between procs:0 and 1 across local face 0
[0]     proc 0  inserting face:0 connection between owner 0 and neighbour 0
[0]     proc 1  inserting face:1 connection between owner 1 and neighbour -2
[1]     proc 0  inserting face:0 connection between owner 0 and neighbour 0
[1]     proc 1  inserting face:1 connection between owner 1 and neighbour -2
[1]     proc 1  inserting face:2 connection between owner 1 and neighbour -2
[0]     proc 1  inserting face:2 connection between owner 1 and neighbour -2
[0]
[0]
[0] --> FOAM FATAL ERROR:
[0] index 9 out of range [0,9]
[0]
[0]     From void Foam::UList<T>::checkIndex(Foam::label) const [with T = Foam::face; Foam::label = int]
[0]     in file /home/pablo/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OpenFOAM/lnInclude/UListI.H at line 138.
[0]
FOAM parallel run aborting
[0]
[1]
[1]
[1] --> FOAM FATAL ERROR:
[1] index 9 out of range [0,9]
[1]
[1]     From void Foam::UList<T>::checkIndex(Foam::label) const [with T = Foam::face; Foam::label = int]
[1]     in file /home/pablo/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OpenFOAM/lnInclude/UListI.H at line 138.
[1]
FOAM parallel run aborting
[0]
[1]
[1]
[1] --> FOAM FATAL ERROR:
[1] index 9 out of range [0,9]
[1]
[1]     From void Foam::UList<T>::checkIndex(Foam::label) const [with T = Foam::face; Foam::label = int]
[1]     in file /home/pablo/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OpenFOAM/lnInclude/UListI.H at line 138.
[1]
[1]
[0] #0  [1] #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&) at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OSspecific/POSIX/printStack/printStack.C:237
 at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OSspecific/POSIX/printStack/printStack.C:237
[1] #1  Foam::error::exitOrAbort(int, bool)[0] #1  Foam::error::exitOrAbort(int, bool) at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OpenFOAM/lnInclude/error.C:265
 at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OpenFOAM/lnInclude/error.C:265
[1] #2  Foam::error::abort()[0] #2  Foam::error::abort() at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OpenFOAM/lnInclude/error.C:307
[0] #3   at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OpenFOAM/lnInclude/error.C:307
[1] #3  Foam::Ostream& Foam::operator<< <Foam::error>(Foam::Ostream&, Foam::errorManip<Foam::error>)Foam::Ostream& Foam::operator<< <Foam::error>(Foam::Ostream&, Foam::errorManip<Foam::error>) in ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/platforms/linux64Gcc75DPInt32Debug/bin/mapFieldsPar
[0] #4   in ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/platforms/linux64Gcc75DPInt32Debug/bin/mapFieldsPar
[1] #4  Foam::UList<Foam::face>::checkIndex(int) constFoam::UList<Foam::face>::checkIndex(int) const in ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/platforms/linux64Gcc75DPInt32Debug/bin/mapFieldsPar
[0] #5  Foam::UList<Foam::face>::operator[](int) in ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/platforms/linux64Gcc75DPInt32Debug/bin/mapFieldsPar
[1] #5  Foam::UList<Foam::face>::operator[](int) at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OpenFOAM/lnInclude/UListI.H:251
 at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/OpenFOAM/lnInclude/UListI.H:251
[0] #6  Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const[1] #6  Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/sampling/meshToMesh/meshToMeshParallelOps.C:860 (discriminator 1)
[0] #7  Foam::meshToMesh::calculate(Foam::word const&, bool) at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/sampling/meshToMesh/meshToMeshParallelOps.C:860 (discriminator 1)
[1] #7  Foam::meshToMesh::calculate(Foam::word const&, bool) at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/sampling/meshToMesh/meshToMesh.C:466 (discriminator 1)
[1] #8  Foam::meshToMesh::constructFromCuttingPatches(Foam::word const&, Foam::word const&, Foam::HashTable<Foam::word, Foam::word, Foam::string::hash> const&, Foam::List<Foam::word> const&, bool) at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/sampling/meshToMesh/meshToMesh.C:466 (discriminator 1)
[0] #8  Foam::meshToMesh::constructFromCuttingPatches(Foam::word const&, Foam::word const&, Foam::HashTable<Foam::word, Foam::word, Foam::string::hash> const&, Foam::List<Foam::word> const&, bool) at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/sampling/meshToMesh/meshToMesh.C:832
 at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/sampling/meshToMesh/meshToMesh.C:832
[1] #9  Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::word const&, Foam::word const&, Foam::HashTable<Foam::word, Foam::word, Foam::string::hash> const&, Foam::List<Foam::word> const&, Foam::meshToMesh::procMapMethod const&, bool)[0] #9  Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::word const&, Foam::word const&, Foam::HashTable<Foam::word, Foam::word, Foam::string::hash> const&, Foam::List<Foam::word> const&, Foam::meshToMesh::procMapMethod const&, bool) at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/sampling/meshToMesh/meshToMesh.C:980
[0] #10   at ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/src/sampling/meshToMesh/meshToMesh.C:980
[1] #10  ?? in ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/platforms/linux64Gcc75DPInt32Debug/bin/mapFieldsPar
[0] #11   in ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/platforms/linux64Gcc75DPInt32Debug/bin/mapFieldsPar
[1] #11  ?? in ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/platforms/linux64Gcc75DPInt32Debug/bin/mapFieldsPar
[0] #12  __libc_start_main in ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/platforms/linux64Gcc75DPInt32Debug/bin/mapFieldsPar
[1] #12  __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
[0] #13   in /lib/x86_64-linux-gnu/libc.so.6
[1] #13  ?? in ~/BuildOpenFoamPlus/OpenFoamGit/openfoam/platforms/linux64Gcc75DPInt32Debug/bin/mapFieldsPar

What is the expected correct behavior?

When running in serial the result is as expected:

mapFieldsPar /decayIsoTurb4.1_copyWithLessTime -sourceTime latestTime
Source: "/home/pablo/Desktop/ApplicationRuhr/Data/paperIlana/twoPointCorrEnric" "decayIsoTurb4.1_copyWithLessTime"
Target: "/home/pablo/Desktop/ApplicationRuhr/Data/paperIlana/twoPointCorrEnric" "decayIsoTurb4.1_trialWithLessElements"

Create databases as time
I/O    : uncollated
Case   : /home/pablo/Desktop/ApplicationRuhr/Data/paperIlana/twoPointCorrEnric/decayIsoTurb4.1_copyWithLessTime
nProcs : 1
Overriding DebugSwitches according to controlDict
    meshToMesh      2;


Source time: 0
Target time: 0

Create meshes

Source mesh size: 2     Target mesh size: 2


Creating and mapping fields for time 0

Creating mesh-to-mesh addressing for region0 and region0 regions using cellVolumeWeight
Source size = 2
Target size = 2
    Overlap volume: 8e-06
    interpolating onto existing field U
ExecutionTime = 0.03 s  ClockTime = 0 s


End

Environment information

  • OpenFOAM version : v2006 and v2012

  • Operating system : Ubuntu 18.04 and OpenSuse 15.2

  • Hardware info :

  • Compiler : gcc7.5

    The "/label bug" text is a gitlab flag that will add the "bug" label to this issue -->

Edited Aug 10, 2021 by Andrew Heather
Assignee
Assign to
Time tracking