[gmx-users] Parallel Use of gromacs - parallel run is 20 times SLOWER than single node run

Yang Ye leafyoung at yahoo.com
Wed Jul 18 08:13:21 CEST 2007


You may explorer more with different number of parallel nodes.

On 7/18/2007 6:33 AM, Jim Kress wrote:
>
> I ran a parallel (mpi) compiled version of gromacs using the following 
> command line:
>
> $ mpirun -np 5 mdrun_mpi -s topol.tpr -np 5 -v
>
> At the end of the file md0.log I found:
>
>         M E G A - F L O P S   A C C O U N T I N G
>
>         Parallel run - timing based on wallclock.
>    RF=Reaction-Field  FE=Free Energy  SCFE=Soft-Core/Free Energy
>    T=Tabulated        W3=SPC/TIP3p    W4=TIP4p (single or pairs)
>    NF=No Forces
>
>  Computing:                        M-Number         M-Flops  % of Flops
> -----------------------------------------------------------------------
>  Coulomb + LJ [W4-W4]            876.631638   234060.647346    88.0
>  Outer nonbonded loop            692.459088     6924.590880     2.6
>  NS-Pairs                        457.344228     9604.228788     3.6
>  Reset In Box                     13.782888      124.045992     0.0
>  Shift-X                         137.773776      826.642656     0.3
>  CG-CoM                            3.445722       99.925938     0.0
>  Sum Forces                      206.660664      206.660664     0.1
>  Virial                           70.237023     1264.266414     0.5
>  Update                           68.886888     2135.493528     0.8
>  Stop-CM                          68.880000      688.800000     0.3
>  P-Coupling                       68.886888      413.321328     0.2
>  Calc-Ekin                        68.893776     1860.131952     0.7
>  Constraint-V                     68.886888      413.321328     0.2
>  Constraint-Vir                   51.675498     1240.211952     0.5
>  Settle                           17.225166     5563.728618     2.1
>  Virtual Site 3                   17.221722      637.203714     0.2
> -----------------------------------------------------------------------
>  Total                                        266063.221098   100.0
> -----------------------------------------------------------------------
>
>                NODE (s)   Real (s)      (%)
>        Time:   3344.000   3344.000    100.0
>                        55:44
>                (Mnbf/s)   (MFlops)   (ns/day)  (hour/ns)
> Performance:      0.262     79.564      0.517     46.444
>
> Detailed load balancing info in percentage of average
> Type        NODE:  0   1   2   3   4 Scaling
> -------------------------------------------
> Coulomb + LJ [W4-W4]:118  94 101 104  80     84%
> Outer nonbonded loop: 97  98  98 103 102     96%
>        NS-Pairs:116  94 101 104  82     85%
>    Reset In Box: 99 100  99 100  99     99%
>         Shift-X: 99 100  99 100  99     99%
>          CG-CoM: 99 100  99 100  99     99%
>      Sum Forces: 99 100  99  99  99     99%
>          Virial: 99 100  99 100  99     99%
>          Update: 99 100  99 100  99     99%
>         Stop-CM: 99 100  99 100  99     99%
>      P-Coupling: 99 100  99 100  99     99%
>       Calc-Ekin: 99 100  99 100  99     99%
>    Constraint-V: 99 100  99 100  99     99%
>  Constraint-Vir: 99 100  99 100  99     99%
>          Settle: 99 100  99 100  99     99%
>  Virtual Site 3: 99 100  99 100  99     99%
>
>     Total Force:118  94 101 104  81     84%
>
>
>     Total Shake: 99 100  99 100  99     99%
>
>
> Total Scaling: 85% of max performance
>
> Finished mdrun on node 0 Sat Jul 14 23:32:32 2007
>
>
> Now, I tried the same calculation on one node and found the following 
> at the end of the file md.log:
>
>         M E G A - F L O P S   A C C O U N T I N G
>
>    RF=Reaction-Field  FE=Free Energy  SCFE=Soft-Core/Free Energy
>    T=Tabulated        W3=SPC/TIP3p    W4=TIP4p (single or pairs)
>    NF=No Forces
>
>  Computing:                        M-Number         M-Flops  % of Flops
> -----------------------------------------------------------------------
>  Coulomb + LJ [W4-W4]            875.182588   233673.750996    88.0
>  Outer nonbonded loop            688.853376     6888.533760     2.6
>  NS-Pairs                        456.997574     9596.949054     3.6
>  Reset In Box                     13.782888      124.045992     0.0
>  Shift-X                         137.773776      826.642656     0.3
>  CG-CoM                            3.445722       99.925938     0.0
>  Virial                           69.156915     1244.824470     0.5
>  Update                           68.886888     2135.493528     0.8
>  Stop-CM                          68.880000      688.800000     0.3
>  P-Coupling                       68.886888      413.321328     0.2
>  Calc-Ekin                        68.893776     1860.131952     0.7
>  Constraint-V                     68.886888      413.321328     0.2
>  Constraint-Vir                   51.675498     1240.211952     0.5
>  Settle                           17.225166     5563.728618     2.1
>  Virtual Site 3                   17.221722      637.203714     0.2
> -----------------------------------------------------------------------
>  Total                                        265406.885286   100.0
> -----------------------------------------------------------------------
>
>                NODE (s)   Real (s)      (%)
>        Time:    165.870    167.000     99.3
>                        2:45
>                (Mnbf/s)   (GFlops)   (ns/day)  (hour/ns)
> Performance:      5.276      1.600     10.418      2.304
> Finished mdrun on node 0 Thu Jul 12 15:17:49 2007
>
>
> While I didn't expect to find pure linear scaling with gromacs.  
> However, I didn't expect to find a massive INCREASE in computational 
> effort across my 5 node, gigabit ethernet cluster.
>
> Anybody understand why this happened?
>
> Thanks.
>
> Jim Kress
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> gmx-users mailing list    gmx-users at gromacs.org
> http://www.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at http://www.gromacs.org/search before posting!
> Please don't post (un)subscribe requests to the list. Use the 
> www interface or send it to gmx-users-request at gromacs.org.
> Can't post? Read http://www.gromacs.org/mailing_lists/users.php




More information about the gromacs.org_gmx-users mailing list