[gmx-users] Fatal Error: cudaFuncGetAttributes failed

[Mayank Vats]

Hi,
I've been getting this error repeatedly when running GROMACS on a cluster:
-------------------------------------------------------
Program:     mdrun_mpi, version 2019.4
Source file: src/gromacs/gpu_utils/gpu_utils.cu (line 100)
MPI rank:    0 (out of 6)

Fatal error:
cudaFuncGetAttributes failed: all CUDA-capable devices are busy or
unavailable

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors
-------------------------------------------------------
The log file is here:
https://drive.google.com/file/d/1pufLp-j02DNhJJJ5Dg2bWEnp_1w4ILk6/view?usp=sharing

Any help would be great!

Thanks,
Mayank Vats
Graduate Student
Chemical and Biological Engineering
Rensselaer Polytechnic Institute
Troy, NY 12180