[gmx-users] Make check failed 2018 Gromacs on GPU workstation

[Phuong Tran]

Thank you for your quick response.

I have no idea why the GPU unit test failed. The workstation was restarted,
so nothing was running at the time.  We have 1 Quadro 600, and 2 1080Ti on
it with CUDA 9.0.
Here is the result for the regression test:

Reading file topol.tpr, VERSION 2018.3 (single precision)

NOTE: Parallelization is limited by the small number of atoms,
      only starting 4 thread-MPI ranks.
      You can use the -nt and/or -ntmpi option to optimize the number of
threads.

Changing nstlist from 10 to 100, rlist from 1.4 to 1.536

Using 4 MPI threads
Using 8 OpenMP threads per tMPI thread


-------------------------------------------------------
Program:     gmx mdrun, version 2018.3
Source file: src/gromacs/taskassignment/taskassignment.cpp (line 251)
Function:    std::vector<std::vector<gmx::GpuTaskMapping> >::value_type
gmx::runTaskAssignment(const std::vector<int>&, const std::vector<int>&,
const gmx_hw_info_t&, const gmx::MDLogger&, const t_commrec*, const
std::vector<gmx::GpuTask>&)
MPI rank:    0 (out of 4)

Inconsistency in user input:
There were 4 GPU tasks found on node ubuntuserver16, but 3 GPUs were
available. If the GPUs are equivalent, then it is usually best to have a
number of tasks that is a multiple of the number of GPUs. You should
reconsider your GPU task assignment, number of ranks, or your use of the
-nb,
-pme, and -npme options, perhaps after measuring the performance you can
get.

On Thu, Sep 13, 2018 at 4:11 PM Szilárd Páll <pall.szilard at gmail.com> wrote:

> Test timeouts are strange, Is the machine you're running on busy with other
> jobs?
>
> Regarding the regressiontest failure, can you share
> tests/regressiontests*/complex/octahedron/mdrun.out
> please?
>
> --
> Szilárd
>
>
> On Thu, Sep 13, 2018 at 8:49 PM Phuong Tran <tranphuonguns at gmail.com>
> wrote:
>
> > Hi all,
> >
> > I have been trying to install Gromacs on our GPU workstation that has
> CUDA
> > 9.0
> > It went well but 'make check' have failed so far on either 2018.3 or
> 2018.2
> > versions.
> > anybody know how to resolve it?
> > Please see the error below.
> > Thank you,
> > -T
> >
> > Test project /home/user/Softwares/gromacs-2018.3/build
> >
> > .....
> >
> >
> >  9/39 Test  #9: GpuUtilsUnitTests ................***Timeout  30.35 sec
> >
> >
> > .....
> >
> >
> > 35/39 Test #35: regressiontests/complex ..........***Failed  174.98 sec
> >
> >
> >
> > GROMACS:      gmx mdrun, version 2018.3
> >
> > Executable:   /home/user/Softwares/gromacs-2018.3/build/bin/gmx
> >
> > Data prefix:  /home/user/Softwares/gromacs-2018.3 (source tree)
> >
> > Working dir:
> > /home/user/Softwares/gromacs-2018.3/build/tests/regressiontests-2018.3
> >
> > Command line:
> >
> >   gmx mdrun -h
> >
> >
> >
> > Thanx for Using GROMACS - Have a Nice Day
> >
> >
> > Mdrun cannot use the requested (or automatic) number of ranks, retrying
> > with 8.
> >
> >
> > Abnormal return value for ' gmx mdrun    -nb cpu   -notunepme >mdrun.out
> > 2>&1' was 1
> >
> > Retrying mdrun with better settings...
> >
> >
> > Abnormal return value for ' gmx mdrun       -notunepme >mdrun.out 2>&1'
> was
> > -1
> >
> > FAILED. Check mdrun.out, md.log file(s) in octahedron for octahedron
> >
> > Re-running orientation-restraints using CPU-based PME
> >
> > Re-running pull_geometry_angle using CPU-based PME
> >
> > Re-running pull_geometry_angle-axis using CPU-based PME
> >
> > Re-running pull_geometry_dihedral using CPU-based PME
> >
> > 1 out of 55 complex tests FAILED
> >
> >
> >  ....
> >
> >
> > 95% tests passed, 2 tests failed out of 39
> >
> >
> > Label Time Summary:
> >
> > GTest              = 133.55 sec (33 tests)
> >
> > IntegrationTest    =  88.38 sec (3 tests)
> >
> > MpiTest            =   2.82 sec (3 tests)
> >
> > UnitTest           =  45.17 sec (30 tests)
> >
> >
> > Total Test time (real) = 810.78 sec
> >
> >
> > The following tests FAILED:
> >
> >   9 - GpuUtilsUnitTests (Timeout)
> >
> >  35 - regressiontests/complex (Failed)
> >
> > Errors while running CTest
> >
> > CMakeFiles/run-ctest-nophys.dir/build.make:57: recipe for target
> > 'CMakeFiles/run-ctest-nophys' failed
> >
> > make[3]: *** [CMakeFiles/run-ctest-nophys] Error 8
> >
> > CMakeFiles/Makefile2:1160: recipe for target
> > 'CMakeFiles/run-ctest-nophys.dir/all' failed
> >
> > make[2]: *** [CMakeFiles/run-ctest-nophys.dir/all] Error 2
> >
> > CMakeFiles/Makefile2:971: recipe for target 'CMakeFiles/check.dir/rule'
> > failed
> >
> > make[1]: *** [CMakeFiles/check.dir/rule] Error 2
> >
> > Makefile:546: recipe for target 'check' failed
> >
> > make: *** [check] Error 2
> > ReplyForward
> > <https://www.google.com/settings/u/0/storage?hl=en>
> > <https://www.google.com/intl/en/policies/terms/>
> > <https://www.google.com/intl/en/policies/privacy/>
> > <https://www.google.com/gmail/about/policy/>
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > send a mail to gmx-users-request at gromacs.org.
> >
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.