[gmx-users] Gromacs 2019 - Ryzen Architecture

[Paul bauer]

Hello,

we only added full detection and support for the newer Rizen chip-sets 
with GROMACS 2019.5, so please try if the update to this version solves 
your issue.
If not, please open an issue on redmine.gromacs.org so we can track the 
problem and try to solve it.

Cheers

Paul

On 02/01/2020 13:26, Sandro Wrzalek wrote:
> Hi,
>
> happy new year!
>
> Now to my problem:
>
> I use Gromacs 2019.3 and to try to run some simulations (roughly 30k 
> atoms per system) on my PC which has the following configuration:
>
> CPU: Ryzen 3950X (overclocked to 4.1 GHz)
>
> GPU #1: Nvidia RTX 2080 Ti
>
> GPU #2: Nvidia RTX 2080 Ti
>
> RAM: 64 GB
>
> PSU: 1600 Watts
>
>
> Each run uses one GPU and 16 of 32 logical cores. Doing only one run 
> at time (gmx mdrun -deffnm rna0 -gpu_id 0 -nb gpu -pme gpu) the GPU 
> utilization is roughly around 84% but if I add a second run, the 
> utilization of both GPUs drops to roughly 20%, while leaving logical 
> cores 17-32 idle (I changed parameter gpu_id, accordingly).
>
> Adding additional parameters for each run:
>
> gmx mdrun -deffnm rna0 -nt 16 -pin on -pinoffset 0 -gpu_id 0 -nb gpu 
> -pme gpu
>
> gmx mdrun -deffnm rna0 -nt 16 -pin on -pinoffset 17 -gpu_id 1 -nb gpu 
> -pme gpu
>
> I get a utilization of 78% per GPU, which is nice but not near the 84% 
> I got with only one run. In theory, however, it should come at least 
> close to that utilization.
>
> I suspect, the Ryzen Chiplet design as the culprit since Gromacs seems 
> to prefer the the first Chiplet, even if two simultaneous simulations 
> have the resources to occupy both. The reason for the 78% utilization 
> could be because of overhead between the two Chiplets via the infinity 
> band. However, I have no proof, nor am I able to explain why gmx mdrun 
> -deffnm rna0 -nt 16 -gpu_id 0 & 1 -nb gpu -pme gpu works as well - 
> seems to occupy free logical cores then.
>
> Long story short:
>
> Are there any workarounds to squeeze the last bit out of my setup? Is 
> it possible to choose the logical cores manually (I did not found 
> anything in the docs so far)?
>
>
> Thank you for your help!
>
>
> Best,
>
> Sandro
>

-- 
Paul Bauer, PhD
GROMACS Development Manager
KTH Stockholm, SciLifeLab
0046737308594