saving . . . saved How do I run mapFields in parallel for OpenFOAM 2.3.0? has been deleted. How do I run mapFields in parallel for OpenFOAM 2.3.0? has been hidden .
How do I run mapFields in parallel for OpenFOAM 2.3.0?
Title
Question
I am using rhoSimpleFoam to simulate a nozzle jet flow. 

But as I increase grid resolution, mapFields takes more and more time. So how do I run mapFields in parallel on 8 processors?

I tried the following command :

mpirun -np 8 mapFields <mySourcedirectory> -parallel

but it just gives an error. 

OpenFOAM General None min None sec 27-08-15, 8:02 a.m. vanihba2000

Answers:

HI Vanihba,

What is the error that you get. We can answer it in a much better way if you tell us the error.
Also did you run decomposePar to decompose the geometry before running it in parallel.

You could try running this command as well : mpirun -np 8 mapFields sourceCase -parallel

Thanks,

Rahul Joshi
27-08-15, 2:22 p.m. rahuljoshi
<span style="line-height: 21.4285717010498px; background-color: rgb(250, 250, 250);">I used </span>decomposePar <span style="line-height: 21.4285717010498px; background-color: rgb(250, 250, 250);">to decompose the geometry. And then ran the command : </span>mpirun -np 8 mapFields <mySourcedirectory> -consistent parallel

This is what I get :

<font size="2">/*---------------------------------------------------------------------------*\\\\\\\\</font>
<font size="2">| ========= | |</font>
<font size="2">| \\\\\\\\\\\\\\\\ / F ield | OpenFOAM: The Open Source CFD Toolbox |</font>
<font size="2">| \\\\\\\\\\\\\\\\ / O peration | Version: 2.3.0 |</font>
<font size="2">| \\\\\\\\\\\\\\\\ / A nd | Web: www.OpenFOAM.org |</font>
<font size="2">| \\\\\\\\\\\\\\\\/ M anipulation | |</font>
<font size="2">\\\\\\\\*---------------------------------------------------------------------------*/</font>
<font size="2">Build : 2.3.0-f5222ca19ce6</font>
<font size="2">Exec : mapFields -consistent ../chevron_v2_2p0_U25/ -parallel</font>
<font size="2">Date : Aug 27 2015</font>
<font size="2">Time : 18:44:37</font>
<font size="2">Host : "admin2-B85M-D3H"</font>
<font size="2">PID : 24985</font>
<font size="2">Case : /home/admin2/Chevron_Work/cases_of/chevron_v2/chevron_U25/chevron_U25_k0.01/chevron_v2_2p5_U25</font>
<font size="2">nProcs : 8</font>
<font size="2">Slaves :</font>
<font size="2">7</font>
<font size="2">(</font>
<font size="2">"admin2-B85M-D3H.24986"</font>
<font size="2">"admin2-B85M-D3H.24987"</font>
<font size="2">"admin2-B85M-D3H.24988"</font>
<font size="2">"admin2-B85M-D3H.24989"</font>
<font size="2">"admin2-B85M-D3H.24990"</font>
<font size="2">"admin2-B85M-D3H.24991"</font>
<font size="2">"admin2-B85M-D3H.24992"</font>
<font size="2">)</font>
<font size="2">
</font>
<font size="2">Pstream initialized with:</font>
<font size="2">floatTransfer : 0</font>
<font size="2">nProcsSimpleSum : 0</font>
<font size="2">commsType : nonBlocking</font>
<font size="2">polling iterations : 0</font>
<font size="2">sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).</font>
<font size="2">fileModificationChecking : Monitoring run-time modified files using timeStampMaster</font>
<font size="2">allowSystemOperations : Disallowing user-supplied system call operations</font>
<font size="2">
</font>
<font size="2">// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //</font>
<font size="2">Source: "../chevron_v2_2p0_U25" ""</font>
<font size="2">Target: "/home/admin2/Chevron_Work/cases_of/chevron_v2/chevron_U25/chevron_U25_k0.01" "chevron_v2_2p5_U25"</font>
<font size="2">
</font>
<font size="2">Create databases as time</font>
<font size="2">Case : ../chevron_v2_2p0_U25</font>
<font size="2">nProcs : 8</font>
<font size="2">Slaves :</font>
<font size="2">7</font>
<font size="2">(</font>
<font size="2">"admin2-B85M-D3H.24986"</font>
<font size="2">"admin2-B85M-D3H.24987"</font>
<font size="2">"admin2-B85M-D3H.24988"</font>
<font size="2">"admin2-B85M-D3H.24989"</font>
<font size="2">"admin2-B85M-D3H.24990"</font>
<font size="2">"admin2-B85M-D3H.24991"</font>
<font size="2">"admin2-B85M-D3H.24992"</font>
<font size="2">)</font>
<font size="2">
</font>
<font size="2">Pstream initialized with:</font>
<font size="2">floatTransfer : 0</font>
<font size="2">nProcsSimpleSum : 0</font>
<font size="2">commsType : nonBlocking</font>
<font size="2">polling iterations : 0</font>
<font size="2">
</font>
<font size="2">Source time: 0</font>
<font size="2">Target time: 0</font>
<font size="2">
</font>
<font size="2">Create meshes</font>
<font size="2">
</font>
<font size="2">Source mesh size: 93771<span class="Apple-tab-span" style="white-space: pre;">\\\\t</span>Target mesh size: 152552</font>
<font size="2">
</font>
<font size="2">
</font>
<font size="2">Consistently creating and mapping fields for time 0</font>
<font size="2">
</font>
<font size="2">Creating mesh-to-mesh addressing for region0 and region0 regions using cellVolumeWeight</font>
<font size="2">[6] #0 Foam::error::printStack(Foam::Ostream&)[2] #0 Foam::error::printStack(Foam::Ostream&)[3] #0 Foam::error::printStack(Foam::Ostream&)[7] #0 Foam::error::printStack(Foam::Ostream&) at ??:?</font>
<font size="2">[6] #1 Foam::sigSegv::sigHandler(int) at ??:?</font>
<font size="2">[7] #1 Foam::sigSegv::sigHandler(int) at ??:?</font>
<font size="2">[2] #1 Foam::sigSegv::sigHandler(int) at ??:?</font>
<font size="2">[3] #1 Foam::sigSegv::sigHandler(int) at ??:?</font>
<font size="2">[2] #2 at ??:?</font>
<font size="2">[3] #2 at ??:?</font>
<font size="2">[6] #2 at ??:?</font>
<font size="2">[7] #2 [5] #0 Foam::error::printStack(Foam::Ostream&) in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[6] #3 cfree in "/lib/x86_6 in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[2] #3 cfree4-linux-gnu/libc.so.6"</font>
<font size="2">[3] #3 cfree[4] [0] #0 Foam::error::printStack(Foam::Ostream&) in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[6] #4 Foam::List<int>::operator=(Foam::List<int> const&) at ??:?</font>
<font size="2">[5] #1 Foam::sigSegv::sigHandler(int)#0 Foam::error::printStack(Foam::Ostream&) in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[7] #3 cfree in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[3] #4 Foam::List<int>::operator=(Foam::List<int> const&) in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[2] #4 Foam::List<int>::operator=(Foam::List<int> const&)[1] #0 Foam::error::printStack(Foam::Ostream&) in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[7] #4 Foam::List<int>::operator=(Foam::List<int> const&) at ??:?</font>
<font size="2">[2] #5 Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const at ??:?</font>
<font size="2">[6] #5 Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const at ??:?</font>
<font size="2">[3] #5 Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const at ??:?</font>
<font size="2">[5] #2 at ??:?</font>
<font size="2">[4] #1 Foam::sigSegv::sigHandler(int) at ??:?</font>
<font size="2">[1] #1 Foam::sigSegv::sigHandler(int) at ??:?</font>
<font size="2">[6] #6 Foam::meshToMesh::calculate() at ??:?</font>
<font size="2">[3] #6 Foam::meshToMesh::calculate() at ??:?</font>
<font size="2">[0] #1 Foam::sigSegv::sigHandler(int) at ??:?</font>
<font size="2">[2] #6 Foam::meshToMesh::calculate() in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[5] #3 cfree at ??:?</font>
<font size="2">[3] #7 Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::meshToMesh::interpolationMethod const&, bool) at ??:?</font>
<font size="2">[2] #7 Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::meshToMesh::interpolationMethod const&, bool) at ??:?</font>
<font size="2">[7] #5 Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const at ??:?</font>
<font size="2">[0] #2 at ??:?</font>
<font size="2">[4] #2 at ??:?</font>
<font size="2">[6] #7 Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::meshToMesh::interpolationMethod const&, bool) at ??:?</font>
<font size="2">[7] #6 Foam::meshToMesh::calculate() in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[5] #4 Foam::List<int>::operator=(Foam::List<int> const&) at ??:?</font>
<font size="2">[1] #2 at ??:?</font>
<font size="2">[2] #8 at ??:?</font>
<font size="2">[3] #8 in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[4] #3 cfree at ??:?</font>
<font size="2">[7] #7 Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::meshToMesh::interpolationMethod const&, bool) at ??:?</font>
<font size="2">[5] #5 Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const at ??:?</font>
<font size="2">[6] #8</font>
<font size="2">in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[0] #3 cfree</font>
<font size="2">in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[4] #4 Foam::List<int>::operator=(Foam::List<int> const&) at ??:?</font>
<font size="2">[5] #6 Foam::meshToMesh::calculate()[2] at ??:?</font>
<font size="2">[2] #9 in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[1] #3 cfree</font>
<font size="2">at ??:?</font>
<font size="2">[7] #8 [3] at ??:?</font>
<font size="2">[3] #9 in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[0] #4 Foam::List<int>::operator=(Foam::List<int> const&) at ??:?</font>
<font size="2">[5] #7 Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::meshToMesh::interpolationMethod const&, bool) in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[1] #4 Foam::List<int>::operator=(Foam::List<int> const&)[6] at ??:?</font>
<font size="2">[6] #9 at ??:?</font>
<font size="2">[4] #5 Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const</font>
<font size="2">at ??:?</font>
<font size="2">[0] #5 Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const at ??:?</font>
<font size="2">[5] #8</font>
<font size="2">
</font>
<font size="2">
</font>
<font size="2">at ??:?</font>
<font size="2">[4] #6 Foam::meshToMesh::calculate() at ??:?</font>
<font size="2">[0] #6 Foam::meshToMesh::calculate()</font>
<font size="2">[6] at ??:?</font>
<font size="2">[6] #10 __libc_start_main at ??:?</font>
<font size="2">[1] #5 Foam::meshToMesh::distributeAndMergeCells(Foam::mapDistribute const&, Foam::polyMesh const&, Foam::globalIndex const&, Foam::Field<Foam::Vector<double> >&, Foam::List<Foam::face>&, Foam::List<int>&, Foam::List<int>&, Foam::List<int>&) const[2] at ??:?</font>
<font size="2">[2] #10 __libc_start_main[3] at ??:?</font>
<font size="2">[3] #10 __libc_start_main[7] at ??:?</font>
<font size="2">[7] #9 [5] at ??:?</font>
<font size="2">[5] #9 in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[2] #11 in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[6] #11 at ??:?</font>
<font size="2">[4] #7 Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::meshToMesh::interpolationMethod const&, bool) at ??:?</font>
<font size="2">[1] #6 Foam::meshToMesh::calculate()</font>
<font size="2">in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[3] #11 at ??:?</font>
<font size="2">[4] #8 at ??:?</font>
<font size="2">[0] #7 Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::meshToMesh::interpolationMethod const&, bool)</font>
<font size="2">
</font>
<font size="2">
</font>
<font size="2">[6] at ??:?</font>
<font size="2">[2] at ??:?</font>
<font size="2">
</font>
<font size="2">
</font>
<font size="2">[7] at ??:?</font>
<font size="2">[7] #10 __libc_start_main at ??:?</font>
<font size="2">[1] #7 Foam::meshToMesh::meshToMesh(Foam::polyMesh const&, Foam::polyMesh const&, Foam::meshToMesh::interpolationMethod const&, bool) at ??:?</font>
<font size="2">[0] #8 [5] at ??:?</font>
<font size="2">[5] #10 __libc_start_main[4] at ??:?</font>
<font size="2">[4] #9</font>
<font size="2">[3] at ??:?</font>
<font size="2">in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[5] #11</font>
<font size="2">at ??:?</font>
<font size="2">[1] #8 [4] at ??:?</font>
<font size="2">[4] #10 __libc_start_main</font>
<font size="2">in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[7] #11</font>
<font size="2">[0] at ??:?</font>
<font size="2">[0] #9 [5] at ??:?</font>
<font size="2">
</font>
<font size="2">[7] at ??:?</font>
<font size="2">
</font>
<font size="2">[1] at ??:?</font>
<font size="2">[1] #9 in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[4] #11</font>
<font size="2">
</font>
<font size="2">[4] at ??:?</font>
<font size="2">[0] at ??:?</font>
<font size="2">[0] #10 __libc_start_main[1] at ??:?</font>
<font size="2">[1] #10 __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[1] #11</font>
<font size="2">in "/lib/x86_64-linux-gnu/libc.so.6"</font>
<font size="2">[0] #11</font>
<font size="2">[0] at ??:?</font>
<font size="2">[1] at ??:?</font>
<font size="2">--------------------------------------------------------------------------</font>
<font size="2">mpirun noticed that process rank 6 with PID 24991 on node admin2-B85M-D3H exited on signal 11 (Segmentation fault).</font>
<font size="2">--------------------------------------------------------------------------</font>
<font size="2">
</font>
27-08-15, 6:54 p.m. vanihba2000
You might have to use the argument "-parallelSource" or "parallelTarget", depending on whether the source or target case is decomposed.
19-06-19, 1:33 p.m. ashish_magar

Login to add comment



27-08-15, 6:52 p.m. vanihba2000


Hi Vanihba,

We will get back to you as soon as possible. Can you tell us the commands that you have used from the beginning for this case.

Thanks,

Rahul Joshi
31-08-15, 11:40 a.m. rahuljoshi
These are the steps and commands I follow and run :

1) First create the case directory with constant and system files. Copy the gmsh created mesh file into the folder.
2) Run gmshToFoam to create polyMesh directory in the constant folder. Then createPatch to create a new polyMesh directory. This I copy into the constant folder and delete the original Polymesh folder.
3) Then, modify the boundary file in the polyMesh directory, to specify the type of the boundary patches.
4) Use decomposePar.
5) Then run mapFields using an old source directory having the same constant and system files, except the polyMesh, of course. (Here I want to run it in parallel).So command : mpirun -np 8 mapFields -consistent <source> -parallel -sourceTime latestTime
6) Since 5 failed, I just run mapFields not in parallel, which takes a lot of time, and thus the 0 directory is created. I modify each file in it to set the boundary field value appropriately.
7) Use decomposePar.
8) Now I just run the case using mpirun -np 8 rhoSimpleFoam -parallel >log.This runs perfectly fine.
31-08-15, 12:38 p.m. vanihba2000

Login to add comment


Log-in to answer to this question.