[gmx-users] GROMACS 2018.2 mdrun GPU Assertion failed: Condition: cudaSuccess == cudaPeekAtLastError()

Jia Hong jiahong96 at hotmail.com
Wed Aug 8 09:53:15 CEST 2018


The log file:


Log file opened on Wed Aug  8 16:33:34 2018
Host: matthias-processing  pid: 28756  rank ID: 0  number of ranks:  1
                      :-) GROMACS - gmx mdrun, 2018.2 (-:

                            GROMACS is written by:
     Emile Apol      Rossen Apostolov      Paul Bauer     Herman J.C. Berendsen
    Par Bjelkmar    Aldert van Buuren   Rudi van Drunen     Anton Feenstra
  Gerrit Groenhof    Aleksei Iupinov   Christoph Junghans   Anca Hamuraru
 Vincent Hindriksen Dimitrios Karkoulis    Peter Kasson        Jiri Kraus
  Carsten Kutzner      Per Larsson      Justin A. Lemkul    Viveca Lindahl
  Magnus Lundborg   Pieter Meulenhoff    Erik Marklund      Teemu Murtola
    Szilard Pall       Sander Pronk      Roland Schulz     Alexey Shvetsov
   Michael Shirts     Alfons Sijbers     Peter Tieleman    Teemu Virolainen
 Christian Wennberg    Maarten Wolf
                           and the project leaders:
        Mark Abraham, Berk Hess, Erik Lindahl, and David van der Spoel

Copyright (c) 1991-2000, University of Groningen, The Netherlands.
Copyright (c) 2001-2017, The GROMACS development team at
Uppsala University, Stockholm University and
the Royal Institute of Technology, Sweden.
check out http://www.gromacs.org for more information.

GROMACS is free software; you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License
as published by the Free Software Foundation; either version 2.1
of the License, or (at your option) any later version.

GROMACS:      gmx mdrun, version 2018.2
Executable:   /home/matthias/Documents/gromacs-2018.2/gpu_install/bin/gmx
Data prefix:  /home/matthias/Documents/gromacs-2018.2/gpu_install
Working dir:  /home/matthias/Documents/MD_simulations/D_EY_ext
Command line:
  gmx mdrun -deffnm md_0_01_GPU

GROMACS version:    2018.2
Precision:          single
Memory model:       64 bit
MPI library:        thread_mpi
OpenMP support:     enabled (GMX_OPENMP_MAX_THREADS = 64)
GPU support:        CUDA
SIMD instructions:  AVX_256
FFT library:        fftw-3.3.8-sse2-avx-avx2-avx2_128-avx512
RDTSCP usage:       enabled
TNG support:        enabled
Hwloc support:      disabled
Tracing support:    disabled
Built on:           2018-08-03 08:06:55
Built by:           matthias at matthias-processing [CMAKE]
Build OS/arch:      Linux 4.15.0-29-generic x86_64
Build CPU vendor:   Intel
Build CPU brand:    Intel(R) Xeon(R) CPU E5-1650 0 @ 3.20GHz
Build CPU family:   6   Model: 45   Stepping: 7
Build CPU features: aes apic avx clfsh cmov cx8 cx16 htt intel lahf mmx msr nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdtscp sse2 sse3 sse4.1 sse4.2 ssse3 tdt x2apic
C compiler:         /usr/bin/cc GNU 5.4.0
C compiler flags:    -mavx     -O3 -DNDEBUG -funroll-all-loops -fexcess-precision=fast
C++ compiler:       /usr/bin/c++ GNU 5.4.0
C++ compiler flags:  -mavx    -std=c++11   -O3 -DNDEBUG -funroll-all-loops -fexcess-precision=fast
CUDA compiler:      /usr/local/cuda-9.0/bin/nvcc nvcc: NVIDIA (R) Cuda compiler driver;Copyright (c) 2005-2017 NVIDIA Corporation;Built on Fri_Sep__1_21:08:03_CDT_2017;Cuda compilation tools, release 9.0, V9.0.176
CUDA compiler flags:-gencode;arch=compute_30,code=sm_30;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_70,code=compute_70;-use_fast_math;-D_FORCE_INLINES;; ;-mavx;-std=c++11;-O3;-DNDEBUG;-funroll-all-loops;-fexcess-precision=fast;
CUDA driver:        9.0
CUDA runtime:       9.0


________________________________
From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se <gromacs.org_gmx-users-bounces at maillist.sys.kth.se> on behalf of Jia Hong <jiahong96 at hotmail.com>
Sent: Wednesday, August 8, 2018 3:50 PM
To: gromacs.org_gmx-users at maillist.sys.kth.se
Subject: [gmx-users] GROMACS 2018.2 mdrun GPU Assertion failed: Condition: cudaSuccess == cudaPeekAtLastError()

Hi,

I've recently compiled gromacs 2018.2 on ubuntu 16.04.5 LTS with CUDA 9.0 on an Nvidia 1080. I've got the error below on the mdrun command. I've searched around (https://mailman-1.sys.kth.se/pipermail/gromacs.org_gmx-users/2018-February/118631.html) and I tried the suggested export CUDA_VISIBLE_DEVICES without any luck.

I currently have 2 nvidia GPUs sitting inside the machine, the first doesn't have CUDA support (old graphics card) while the second is the aforementioned 1080. I've also verified that the 1080 has gpu id 1, and ran the command:

  gmx mdrun -deffnm md_0_01_GPU -gpu_id 1

which gives the same error.
I've attached the log file too.

(I don't know if this helps, but I manage to compile a CUDA test program to verify that my CUDA installation isn't broken: http://ramblingsofagamedevstudent.blogspot.com/2014/03/set-up-visual-studio-2012-for-cuda.html)

Cheers,
Harry

The output:


:-) GROMACS - gmx mdrun, 2018.2 (-:

                            GROMACS is written by:
     Emile Apol      Rossen Apostolov      Paul Bauer     Herman J.C. Berendsen
    Par Bjelkmar    Aldert van Buuren   Rudi van Drunen     Anton Feenstra
  Gerrit Groenhof    Aleksei Iupinov   Christoph Junghans   Anca Hamuraru
 Vincent Hindriksen Dimitrios Karkoulis    Peter Kasson        Jiri Kraus
  Carsten Kutzner      Per Larsson      Justin A. Lemkul    Viveca Lindahl
  Magnus Lundborg   Pieter Meulenhoff    Erik Marklund      Teemu Murtola
    Szilard Pall       Sander Pronk      Roland Schulz     Alexey Shvetsov
   Michael Shirts     Alfons Sijbers     Peter Tieleman    Teemu Virolainen
 Christian Wennberg    Maarten Wolf
                           and the project leaders:
        Mark Abraham, Berk Hess, Erik Lindahl, and David van der Spoel

Copyright (c) 1991-2000, University of Groningen, The Netherlands.
Copyright (c) 2001-2017, The GROMACS development team at
Uppsala University, Stockholm University and
the Royal Institute of Technology, Sweden.
check out http://www.gromacs.org for more information.

GROMACS is free software; you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License
as published by the Free Software Foundation; either version 2.1
of the License, or (at your option) any later version.

GROMACS:      gmx mdrun, version 2018.2
Executable:   /home/matthias/Documents/gromacs-2018.2/gpu_install/bin/gmx
Data prefix:  /home/matthias/Documents/gromacs-2018.2/gpu_install
Working dir:  /home/matthias/Documents/MD_simulations/D_EY_ext
Command line:
  gmx mdrun -deffnm md_0_01_GPU


Back Off! I just backed up md_0_01_GPU.log to ./#md_0_01_GPU.log.10#

-------------------------------------------------------
Program:     gmx mdrun, version 2018.2
Source file: src/gromacs/gpu_utils/gpu_utils.cu (line 762)
Function:    void findGpus(gmx_gpu_info_t*)

Assertion failed:
Condition: cudaSuccess == cudaPeekAtLastError()
We promise to return with clean CUDA state!

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors



More information about the gromacs.org_gmx-users mailing list