[gmx-users] Running GPU issue
Kovalskyy, Dmytro
Kovalskyy at uthscsa.edu
Thu Nov 15 20:54:01 CET 2018
The error you saw is clear evidence of a bug. There's a few things you
might try to help narrow things down.
* Is the error reproducible on each run of mdrun?
Yes
* 2018.3 was not designed or tested on CUDA 10, which was released rather
later. If you have an earlier CUDA, please see if building with it
alleviates the error
No I have only CUDA 10.0 installed. Taking into account that I made that running (see replies to the next questions) please let me know whether should I go with CUDA 9.
* The error message could be triggered by multiple parts of the code; what
do you get with mdrun -nb gpu -pme cpu?
This way I get MD running, no crash. All 72 cores (dual Xeon Gold 6140)
with my system I got
$mdrun -deffnm md -v -nb gpu -pme cpu
(ns/day) (hour/ns)
Performance: 56.867 0.422
$nvidia-smi
| 0 Quadro P5000 On | 00000000:D5:00.0 On | Off |
| 35% 56C P0 63W / 180W | 1400MiB / 16256MiB | 37% Default |
* Do you get any more diagnostics from running with a build with cmake
-DCMAKE_BUILD_TYPE=Debug?
Interesting comes here. Gromacs with Debug = ON allows to stably run MD with no additional params.
I mean
$ gmx mdrun -deffnm md -v
runs with no crash. However, only half of the Xeon cores are used (monitored with htop)
(ns/day) (hour/ns)
Performance: 69.577 0.345
$nvidia-smi
| 0 Quadro P5000 On | 00000000:D5:00.0 On | Off |
| 38% 62C P0 68W / 180W | 1430MiB / 16256MiB | 56% Default |
If I run
$ gmx mdrun -deffnm md -v -nb gpu -pme cpu
Then all CPU cores are busy but performancs is poor
(ns/day) (hour/ns)
Performance: 20.249 1.185
$nvidia-smi
| 0 Quadro P5000 On | 00000000:D5:00.0 On | Off |
| 37% 57C P0 58W / 180W | 1408MiB / 16256MiB | 22% Default |
Finally,
If I run same MD with no GPU at all (i.e. -nb cpu) then I got performance
(ns/day) (hour/ns)
Performance: 47.536 0.505
Does this help?
Dmytro
From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se <gromacs.org_gmx-users-bounces at maillist.sys.kth.se> on behalf of Mark Abraham <mark.j.abraham at gmail.com>
Sent: Wednesday, November 14, 2018 1:37 PM
To: gmx-users at gromacs.org
Cc: gromacs.org_gmx-users at maillist.sys.kth.se
Subject: Re: [gmx-users] Running GPU issue
Hi,
I expect that that warning is fine (for now).
The error you saw is clear evidence of a bug. There's a few things you
might try to help narrow things down.
* Is the error reproducible on each run of mdrun?
* 2018.3 was not designed or tested on CUDA 10, which was released rather
later. If you have an earlier CUDA, please see if building with it
alleviates the error
* The error message could be triggered by multiple parts of the code; what
do you get with mdrun -nb gpu -pme cpu?
* Do you get any more diagnostics from running with a build with cmake
-DCMAKE_BUILD_TYPE=Debug?
Mark
On Wed, Nov 14, 2018 at 1:09 PM Kovalskyy, Dmytro <Kovalskyy at uthscsa.edu>
wrote:
> I forgot to add. While compiling gromacs I got following error at the very
> beggining:
>
>
> [ 3%] Built target gpu_utilstest_cuda
> /usr/local/tmp/gromacs-2018.3/src/gromacs/gpu_utils/gpu_utils.cu: In
> function ?int do_sanity_checks(int, cudaDeviceProp*)?:
> /usr/local/tmp/gromacs-2018.3/src/gromacs/gpu_utils/gpu_utils.cu:258:28:
> warning: ?cudaError_t cudaThreadSynchronize()? is deprecated
> [-Wdeprecated-declarations]
> if (cudaThreadSynchronize() != cudaSuccess)
> ^
> /usr/local/cuda/include/cuda_runtime_api.h:947:46: note: declared here
> extern __CUDA_DEPRECATED __host__ cudaError_t CUDARTAPI
> cudaThreadSynchronize(void);
>
>
> But make has completed its job without falling down.
>
>
>
>
> ________________________________________
> From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se <
> gromacs.org_gmx-users-bounces at maillist.sys.kth.se> on behalf of Mark
> Abraham <mark.j.abraham at gmail.com>
> Sent: Tuesday, November 13, 2018 10:29 PM
> To: gmx-users at gromacs.org
> Cc: gromacs.org_gmx-users at maillist.sys.kth.se
> Subject: Re: [gmx-users] Running GPU issue
>
> Hi,
>
> It can share.
>
> Mark
>
> On Mon, Nov 12, 2018 at 10:19 PM Kovalskyy, Dmytro <Kovalskyy at uthscsa.edu>
> wrote:
>
> > Hi,
> >
> >
> >
> > To perform GPU with Gromacs does it require exclusive GPU card or
> Gromacs
> > can share the video card with X-server?
> >
> >
> > Thank you
> >
> >
> > Dmytro
> > --
> > Gromacs Users mailing list
> >
> > * Please search the archive at
> > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> > posting!
> >
> > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
> >
> > * For (un)subscribe requests visit
> > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> > send a mail to gmx-users-request at gromacs.org.
> >
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>
--
Gromacs Users mailing list
* Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!
* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
* For (un)subscribe requests visit
https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.
More information about the gromacs.org_gmx-users
mailing list