[gmx-users] GROMACS 2018.2 mdrun GPU Assertion failed: Condition: cudaSuccess == cudaPeekAtLastError()

[Jia Hong]

Hi,

I've recently compiled gromacs 2018.2 on ubuntu 16.04.5 LTS with CUDA 9.0 on an Nvidia 1080. I've got the error below on the mdrun command. I've searched around (https://mailman-1.sys.kth.se/pipermail/gromacs.org_gmx-users/2018-February/118631.html) and I tried the suggested export CUDA_VISIBLE_DEVICES without any luck.

I currently have 2 nvidia GPUs sitting inside the machine, the first doesn't have CUDA support (old graphics card) while the second is the aforementioned 1080. I've also verified that the 1080 has gpu id 1, and ran the command:

  gmx mdrun -deffnm md_0_01_GPU -gpu_id 1

which gives the same error.
I've attached the log file too.

(I don't know if this helps, but I manage to compile a CUDA test program to verify that my CUDA installation isn't broken: http://ramblingsofagamedevstudent.blogspot.com/2014/03/set-up-visual-studio-2012-for-cuda.html)

Cheers,
Harry

The output:


:-) GROMACS - gmx mdrun, 2018.2 (-:

                            GROMACS is written by:
     Emile Apol      Rossen Apostolov      Paul Bauer     Herman J.C. Berendsen
    Par Bjelkmar    Aldert van Buuren   Rudi van Drunen     Anton Feenstra
  Gerrit Groenhof    Aleksei Iupinov   Christoph Junghans   Anca Hamuraru
 Vincent Hindriksen Dimitrios Karkoulis    Peter Kasson        Jiri Kraus
  Carsten Kutzner      Per Larsson      Justin A. Lemkul    Viveca Lindahl
  Magnus Lundborg   Pieter Meulenhoff    Erik Marklund      Teemu Murtola
    Szilard Pall       Sander Pronk      Roland Schulz     Alexey Shvetsov
   Michael Shirts     Alfons Sijbers     Peter Tieleman    Teemu Virolainen
 Christian Wennberg    Maarten Wolf
                           and the project leaders:
        Mark Abraham, Berk Hess, Erik Lindahl, and David van der Spoel

Copyright (c) 1991-2000, University of Groningen, The Netherlands.
Copyright (c) 2001-2017, The GROMACS development team at
Uppsala University, Stockholm University and
the Royal Institute of Technology, Sweden.
check out http://www.gromacs.org for more information.

GROMACS is free software; you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License
as published by the Free Software Foundation; either version 2.1
of the License, or (at your option) any later version.

GROMACS:      gmx mdrun, version 2018.2
Executable:   /home/matthias/Documents/gromacs-2018.2/gpu_install/bin/gmx
Data prefix:  /home/matthias/Documents/gromacs-2018.2/gpu_install
Working dir:  /home/matthias/Documents/MD_simulations/D_EY_ext
Command line:
  gmx mdrun -deffnm md_0_01_GPU


Back Off! I just backed up md_0_01_GPU.log to ./#md_0_01_GPU.log.10#

-------------------------------------------------------
Program:     gmx mdrun, version 2018.2
Source file: src/gromacs/gpu_utils/gpu_utils.cu (line 762)
Function:    void findGpus(gmx_gpu_info_t*)

Assertion failed:
Condition: cudaSuccess == cudaPeekAtLastError()
We promise to return with clean CUDA state!

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors