[gmx-users] Can we convert one job into many serial jobs?

[Christopher Neale]

Sounds like you are asking for packed nodes on a cluster that allows jobs to take only a single core of a full node. You should do some benchmarking to see what performance you get when you allow your mpi job to use distributed cores (rather than packed nodes). You'll need to run a few repeats of the tests with distributed cores because if your IB is insufficient then your mpi speed with distributed cores will vary widely based on the particular way your cores are distributed across the cluster. I use clusters in which distributed cores give horrendous performance, and others where there is only about a 2% performance hit compared to packed-node jobs. I agree with Mark that the use of distributed cores an an infiniband cluster is a bad idea, but I've never succeeded in convincing  anyone to change their setup so complaining may be a waste of your time. It might be better to provide the benchmarking data and ask them if there is anything they can do to help you (if indeed your jobs scale poorly when using distributed cores). It is within the power of a system administrator to create a reservation for your jobs. Nobody likes to do this because if your jobs crash or finish then the nodes sit idle, but in some cases it is the only option (and in fact it can be more efficient for the cluster because there is no need to backfill while nodes are being added to your reservation by the queueing system).

Chris.

________________________________________
From: gromacs.org_gmx-users-bounces at maillist.sys.kth.se <gromacs.org_gmx-users-bounces at maillist.sys.kth.se> on behalf of Zhang, Cheng <c.zhang.11 at ucl.ac.uk>
Sent: 25 May 2015 10:07
To: gromacs.org_gmx-users at maillist.sys.kth.se
Cc: Zhang, Cheng
Subject: [gmx-users] Can we convert one job into many serial jobs?

Dear GROMACS,

I am now using openmpi nodes to run GROMACS (e.g. mdrun_mpi) on our cluster. When the nodes required are too many (e.g. more than 8), jobs always take a long time to wait in the queue. So I wonder if there is a possibility that we can


1) convert the job into many serial jobs?

2) convert the job into several openmpi jobs but with only 1 node in each job?


When serial jobs are submitted, hundreds of jobs can be run within just a few minutes.


Thank you very much.


Yours sincerely

Cheng
--
Gromacs Users mailing list

* Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting!

* Can't post? Read http://www.gromacs.org/Support/Mailing_Lists

* For (un)subscribe requests visit
https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-request at gromacs.org.