[gmx-users] Domain decomposition error while running coarse grained simulations on cluster

Thu Sep 5 04:02:01 CEST 2019

On 9/3/19 12:21 PM, Avijeet Kulshrestha wrote:
> Hi Justin,
> Thanks for replying to my query. Please see the error message below of log
> file.
> I have user-defined bonded, angle potential which I am providing by
> supplying tabulated data. I have position restraint also on backbone atoms
> of the protein that is only in the equilibration steps.
>
> *This is the error message of the log file. *
> Initializing Domain Decomposition on 8 ranks
> Dynamic load balancing: off
> Minimum cell size due to atom displacement: 0.546 nm
> Initial maximum inter charge-group distances:
>      two-body bonded interactions: 12.145 nm, LJ-14, atoms 11 568

Here's your problem. You have pairs defined that are in excess of 12 nm, 
but they are assigned to a 1-4 interaction, so atoms that should be 
separated by three bonds. The user-defined potential shouldn't matter 
here unless you've added [pairs] to the topology.

-Justin

>    multi-body bonded interactions: 1.124 nm, G96Angle, atoms 3767 3770
> Minimum cell size due to bonded interactions: 13.359 nm
> Maximum distance for 5 constraints, at 120 deg. angles, all-trans: 1.360 nm
> Estimated maximum distance required for P-LINCS: 1.360 nm
> Scaling the initial minimum size with 1/0.8 (option -dds) = 1.25
> Optimizing the DD grid for 8 cells with a minimum initial size of 16.699 nm
> The maximum allowed number of cells is: X 0 Y 0 Z 1
>
> -------------------------------------------------------
> Program:     gmx mdrun, version 2018.6
> Source file: src/gromacs/domdec/domdec.cpp (line 6594)
> MPI rank:    0 (out of 8)
>
> Fatal error:
> There is no domain decomposition for 8 ranks that is compatible with the
> given
> box and a minimum cell size of 16.6989 nm
> Change the number of ranks or mdrun option -rdd or -dds
> Look in the log file for details on the domain decomposition
>
> Please let me know what I can do to rectify it.
>
> On Mon, 2 Sep 2019 at 12:25, <
> gromacs.org_gmx-users-request at maillist.sys.kth.se> wrote:
>
>> Send gromacs.org_gmx-users mailing list submissions to
>>          gromacs.org_gmx-users at maillist.sys.kth.se
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>          https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users
>> or, via email, send a message with subject or body 'help' to
>>          gromacs.org_gmx-users-request at maillist.sys.kth.se
>>
>> You can reach the person managing the list at
>>          gromacs.org_gmx-users-owner at maillist.sys.kth.se
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of gromacs.org_gmx-users digest..."
>>
>>
>> Today's Topics:
>>
>>     1. Re: simulation termination problem (Prabir Khatua)
>>     2. Re: wham analysis (Justin Lemkul)
>>     3. Re: Domain decomposition error while running coarse grained
>>        simulations on cluster (Justin Lemkul)
>>     4. Re: mdrun error (Justin Lemkul)
>>     5. regarding changing the scale from ps to ns (sudha bhagwati)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Sun, 1 Sep 2019 12:10:04 -0500
>> From: Prabir Khatua <prabir07chem at gmail.com>
>> To: gmx-users at gromacs.org
>> Subject: Re: [gmx-users] simulation termination problem
>> Message-ID:
>>          <
>> CAObZXoOgMnZMKy8Ddreyhfh7CjM9yCPpo3fg6DMOE5fN-mgx0g at mail.gmail.com>
>> Content-Type: text/plain; charset="UTF-8"
>>
>> Thanks Justin. The problem has been fixed.
>>
>> On Fri, Aug 30, 2019 at 7:31 AM Justin Lemkul <jalemkul at vt.edu> wrote:
>>
>>>
>>> On 8/29/19 12:31 PM, Prabir Khatua wrote:
>>>> Hello Gromacs users,
>>>>
>>>> I am trying to simulate a system of atom size 3,58,973 in gromacs
>> 5.1.5.
>>>> However, my simulation is being terminated in between with the
>> following
>>>> error.
>>>>
>>>> File input/output error:
>>>> Cannot rename checkpoint file; maybe you are out of disk space?
>>>> For more information and tips for troubleshooting, please check the
>>> GROMACS
>>>> website at http://www.gromacs.org/Documentation/Errors
>>>>
>>>> I did not find any solution with respect to the error I was having in
>> the
>>>> mentioned website. What I found was related to memory issue. I do not
>>> know
>>>> whether this is the same issue.
>>> The issue is not related to memory, it is (potentially) related to disk
>>> space. Do you have enough space on the filesystem to write output files?
>>> This can also happen sometimes when the filesystem blips. There's not
>>> much you can do about that except complain to your sysadmin about
>>> integrity of the filesystem.
>>>
>>> -Justin
>>>
>>>> Please note that I was successfully able to run another simulation of a
>>>> system having relatively less number of atoms with same script. The run
>>>> command that I used for the simulation was
>>>>
>>>> mpirun -np 48 gmx_mpi mdrun -ntomp 1 -deffnm npt
>>>>
>>>> I ran both the simulations on two nodes having 24 cpu cores in each one
>>> of
>>>> the nodes.
>>>> I am also not able to figure out one issue. The log file of the system
>>>> where the simulation was successfully completed showed
>>>>
>>>> Running on 2 nodes with total 48 cores, 48 logical cores, 0 compatible
>>> GPUs
>>>>     Cores per node:           24
>>>>     Logical cores per node:   24
>>>>     Compatible GPUs per node:  0
>>>>
>>>> However, in the unsuccessful case, the log file showed
>>>>
>>>> Running on 1 node with total 24 cores, 24 logical cores
>>>>
>>>> Thus, it looks like the simulation was running on single node although
>> I
>>>> asked it to run on two nodes. I have no idea how to fix this issue.
>>> Please
>>>> help me fix this issue or what I am doing wrong.
>>>>
>>>> Thanks in advance,
>>>>
>>>> Prabir
>>>>
>>> --
>>> ==================================================
>>>
>>> Justin A. Lemkul, Ph.D.
>>> Assistant Professor
>>> Office: 301 Fralin Hall
>>> Lab: 303 Engel Hall
>>>
>>> Virginia Tech Department of Biochemistry
>>> 340 West Campus Dr.
>>> Blacksburg, VA 24061
>>>
>>> jalemkul at vt.edu | (540) 231-3129
>>> http://www.thelemkullab.com
>>>
>>> ==================================================
>>>
>>> --
>>> Gromacs Users mailing list
>>>
>>> * Please search the archive at
>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>>> posting!
>>>
>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>>
>>> * For (un)subscribe requests visit
>>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>>> send a mail to gmx-users-request at gromacs.org.
>>>
>>
>> --
>>
>> *Prabir Khatua*
>> *Postdoctoral Research Associate*
>> *Department of Chemistry & Biochemistry*
>> *University of Oklahoma*
>> *Norman, Oklahoma 73019*
>> *U. S. A.*
>>
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Sun, 1 Sep 2019 13:21:23 -0400
>> From: Justin Lemkul <jalemkul at vt.edu>
>> To: gmx-users at gromacs.org
>> Subject: Re: [gmx-users] wham analysis
>> Message-ID: <65191dc8-43fd-bc2f-6b9a-20903052df32 at vt.edu>
>> Content-Type: text/plain; charset=utf-8; format=flowed
>>
>>
>>
>> On 9/1/19 5:35 AM, Negar Parvizi wrote:
>>>    Dear all,
>>> I used Justin tutorial(Tutorial 3: Umbrella Sampling: GROMACS Tutorial )
>> for my file which is protein-ligand complex.
>>> The pulling force was in Y direction. when Umbrella sampling finished,
>> "Wham" couldn't analysis the data because wham is in z direction.what
>> should I do now for wham analysis? how can I change it to Y direction?
>>> I sent message in gromacs comunity , what Justin said:
>>> "WHAM does not presuppose the axis or vector; it does what you tell it.
>> If you're referring to the x-axis label in the PMF profile being "z,"  that
>> is just a generic (and perhaps imprecise) label that should be  changed to
>> the Greek character xi, per conventional notation."
>>> I didn't understand it.
>> I made a guess based on minimal information. You asserted that you
>> pulled along y but WHAM indicated the bias was along z. I know that the
>> default x-axis label in profile.xvg says "z" and it causes confusion. So
>> I provided the comment that I did.
>>
>> However, it is clear from the gmx wham output below that you did *not*
>> apply a bias along y, as you stated:
>>
>>> File umbrella0.tpr, 1 coordinates, geometry "distance", dimensions [N N
>> Y], (1 dimensions)
>>
>> This means your reaction coordinate was the z-axis.
>>
>>> So I decided copy the error:
>>>
>>> Here is the error:
>>>
>>>> Found 25 tpr and 25 pull force files in tpr-files.dat and
>> pullf-files.dat, respectively
>>>> Reading 12 tpr and pullf files
>>>> Automatic determination of boundaries...
>>>> Reading file umbrella0.tpr, VERSION 5.1.4 (single precision)
>>>> File umbrella0.tpr, 1 coordinates, geometry "distance", dimensions [N N
>> Y], (1 dimensions)
>>>     Pull group coordinates not expected in pullx files.
>>>     crd 0) k = 1000   position = 0.840198
>>>     Use option -v to see this output for all input tpr files
>>>
>>>> Reading pull force file with pull geometry distance and 1 pull
>> dimensions
>>>> Expecting these columns in pull file:
>>>       0 reference columns for each individual pull coordinate
>>>       1 data columns for each pull coordinate
>>>> With 1 pull groups, expect 2 columns (including the time column)
>>>> Reading file umbrella71.tpr, VERSION 5.1.4 (single precision)
>>>> Reading file umbrella98.tpr, VERSION 5.1.4 (single precision)
>>>> Reading file umbrella111.tpr, VERSION 5.1.4 (single precision)
>>>> Reading file umbrella119.tpr, VERSION 5.1.4 (single precision)
>>>> Reading file umbrella139.tpr, VERSION 5.1.4 (single precision)
>>>> Reading file umbrella146.tpr, VERSION 5.1.4 (single precision)
>>>> Reading file umbrella157.tpr, VERSION 5.1.4 (single precision)
>>>> Reading file umbrella180.tpr, VERSION 5.1.4 (single precision)
>>>> Reading file umbrella202.tpr, VERSION 5.1.4 (single precision)
>> What happens next? Nothing here says "error," however it looks like your
>> input files are of an unexpected format. Perhaps you've switched
>> pullx.xvg and pullf.xvg. As someone else suggested, you can use
>> pullf.xvg files to get the PMF profile and avoid the issue entirely.
>>
>> -Justin
>>
>> --
>> ==================================================
>>
>> Justin A. Lemkul, Ph.D.
>> Assistant Professor
>> Office: 301 Fralin Hall
>> Lab: 303 Engel Hall
>>
>> Virginia Tech Department of Biochemistry
>> 340 West Campus Dr.
>> Blacksburg, VA 24061
>>
>> jalemkul at vt.edu | (540) 231-3129
>> http://www.thelemkullab.com
>>
>> ==================================================
>>
>>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Sun, 1 Sep 2019 13:22:15 -0400
>> From: Justin Lemkul <jalemkul at vt.edu>
>> To: gmx-users at gromacs.org
>> Subject: Re: [gmx-users] Domain decomposition error while running
>>          coarse grained simulations on cluster
>> Message-ID: <71840e02-8747-38b0-d2a0-39da5adbb404 at vt.edu>
>> Content-Type: text/plain; charset=utf-8; format=flowed
>>
>>
>>
>> On 9/1/19 5:44 AM, Avijeet Kulshrestha wrote:
>>> Hi all,
>>> I am running martini coarse-grained simulation with 15 fs of time step in
>>> gromacs 2018.6. I have 25859 number of atoms and my box size is:
>>> 12.00000  14.00000  18.00000
>>> Where I have Protein, membrane (DPPC) and ions.
>>> I have minimized energy with 16 processor and -rdd option as 2.5. It
>> worked
>>> fine but later in NVT simulation, this started giving errors.
>>> *I am getting the following error without -rdd option when I used 1 node
>>> and 8 processor per node:*
>>> Fatal error:
>>> There is no domain decomposition for 8 ranks that is compatible with the
>>> given
>>> box and a minimum cell size of 7.69783 nm
>> This minimum cell size is significantly larger than a normal simulation
>> would require. Do you have restraints or other non-standard interactions
>> defined? The .log file snippet that describes the DD setup would be
>> useful for tracking the issue down.
>>
>> -Justin
>>
>> --
>> ==================================================
>>
>> Justin A. Lemkul, Ph.D.
>> Assistant Professor
>> Office: 301 Fralin Hall
>> Lab: 303 Engel Hall
>>
>> Virginia Tech Department of Biochemistry
>> 340 West Campus Dr.
>> Blacksburg, VA 24061
>>
>> jalemkul at vt.edu | (540) 231-3129
>> http://www.thelemkullab.com
>>
>> ==================================================
>>
>>
>>
>> ------------------------------
>>
>> Message: 4
>> Date: Sun, 1 Sep 2019 13:23:48 -0400
>> From: Justin Lemkul <jalemkul at vt.edu>
>> To: gmx-users at gromacs.org
>> Subject: Re: [gmx-users] mdrun error
>> Message-ID: <a3299ee2-b645-a330-65f8-3bd18d65a3ed at vt.edu>
>> Content-Type: text/plain; charset=utf-8; format=flowed
>>
>>
>>
>> On 9/1/19 7:02 AM, m g wrote:
>>> Dear Justin,
>> There are many other people on this list, and I don't always have answers
>> :)
>>
>>>    I'm simulating a system with "wall" for z direction, but I gave this
>> error in minimization step "software inconsistency error: lost particles
>> while sorting",.Would you please help me? I used the following?parameter:
>>
>> What GROMACS version are you using? CPU or GPU? Does the simulation work
>> without the walls? You need to provide a lot more diagnostic
>> information. This is a cryptic error that suggests something very
>> fundamental is wrong. That also makes it very hard to diagnose.
>>
>> If you're not using the latest version of GROMACS, start there. I recall
>> some bug fixes that may be relevant. If the error is reproducible in
>> 2019.3, please post a more complete description of what you're doing.
>>
>> -Justin
>>
>>> integrator? ??? =steep
>>>
>>> emtol?????? ?? ?????= 100.0emstep????????????? = 0.01nsteps???????????
>> ??? = 5000nstlist?????????? ??? = 1cutoff-scheme?????? =
>> Verletns_type?????????? ??? = gridrlist?????? ?? ?????= 1.2? ??coulombtype
>> ?? ?????= PMErcoulomb??? ?? ?????= 1.2vdwtype???????????? =
>> cutoffvdw-modifier??????? = force-switchrvdw-switch???????? =
>> 1.0rvdw??????? ? ??????= 1.2????????? ??DispCorr??????????? = nopbc ?
>> ??????????????=xy? ? ? ? ? ? ? ? ? ??nwall?????????????? =
>> 2wall_type?????????? = 10-4wall_r_linpot?????? = 1wall_atomtype?????? = ca?
>> cawall_density??????? = 60? 60wall_ewald_zfac???? = 3ewald-geometry????? =
>> 3dc
>>> Thanks,Ganj
>>>
>> --
>> ==================================================
>>
>> Justin A. Lemkul, Ph.D.
>> Assistant Professor
>> Office: 301 Fralin Hall
>> Lab: 303 Engel Hall
>>
>> Virginia Tech Department of Biochemistry
>> 340 West Campus Dr.
>> Blacksburg, VA 24061
>>
>> jalemkul at vt.edu | (540) 231-3129
>> http://www.thelemkullab.com
>>
>> ==================================================
>>
>>
>>
>> ------------------------------
>>
>> Message: 5
>> Date: Mon, 2 Sep 2019 12:23:25 +0530
>> From: sudha bhagwati <sudhabhagwati at gmail.com>
>> To: gmx-users at gromacs.org
>> Subject: [gmx-users] regarding changing the scale from ps to ns
>> Message-ID:
>>          <CAAjYSb-58-5qEeM+Rq-Yv3yTApA8cb1ktL_DP74n8MoK3j=
>> 34g at mail.gmail.com>
>> Content-Type: text/plain; charset="UTF-8"
>>
>> Dear gmx users,
>>
>> I am using the GROAMCS 5.1.4 version.
>>
>> ----------------------------------------------------------------------------------------------------------------------------------------
>> Running the commands below:
>>
>>
>> *gmx gyrate -s md_0_20.tpr -f md_0_20_noPBC.xtc -o gyrate.xvg -tu ns*
>> *OR*
>> *gmx energy -f ie.edr -o interaction_energy.xvg -tu ns*
>>
>>
>> Error while executing commands:
>>
>> *Error in user input:*
>>
>> *Invalid command-line options    Unknown command-line option -tu*
>>
>> ----------------------------------------------------------------------------------------------------------------------------------------
>>
>> -tu flag is not working for commands showed above. I want to generate my
>> plots for nanosecond scale using xmgrace. Could you please help me out with
>> this issue. I'd be really grateful to you.
>>
>> Thank you.
>>
>>
>> --
>> Thanks & regards
>> ~
>> Sudha Bhagwati
>>
>>
>> ------------------------------
>>
>> --
>> Gromacs Users mailing list
>>
>> * Please search the archive at
>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
>> posting!
>>
>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>>
>> * For (un)subscribe requests visit
>> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
>> send a mail to gmx-users-request at gromacs.org.
>>
>> End of gromacs.org_gmx-users Digest, Vol 185, Issue 3
>> *****************************************************
>>

-- 
==================================================

Justin A. Lemkul, Ph.D.
Assistant Professor
Office: 301 Fralin Hall
Lab: 303 Engel Hall

Virginia Tech Department of Biochemistry
340 West Campus Dr.
Blacksburg, VA 24061

jalemkul at vt.edu | (540) 231-3129
http://www.thelemkullab.com

==================================================