[gmx-developers] Please cancel outdated CI pipelines

[Paul bauer]

For those that are interested, here is the MR:

https://gitlab.com/gromacs/gromacs/-/merge_requests/594

/Paul

On 28/09/2020 17:15, Paul Bauer wrote:
> Hello,
>
> I started to set up tests that can run as jobs after a commit is 
> merged, and we should just see that we get this code in to reduce the 
> stress on the hardware. All the slow jobs can then be moved there.
>
> /Paul
>
> On Mon, 28 Sep 2020, 17:00 Erik Lindahl, <erik.lindahl at gmail.com 
> <mailto:erik.lindahl at gmail.com>> wrote:
>
>     Hi,
>
>     Good point, but it also shows we have some homework to do. Our new
>     CI infrastructure was quite expanded (~80 high end cores,
>     2GB/core, all SSD disks) - but this only seems to have led all of
>     us to happily add tests that took more time.
>
>     Looking just briefly at the pipelines, it seems the testing phase
>     is our main culprit. While it is of course nice to have per-change
>     tests, I don't think it's sustainable that we need 10+ CPU hours
>     of testing for every typo fix.
>
>     In particular these tests need attention:
>
>     - gmx-api. They both take 12-15 minutes on two cores, and there
>     are four of them.
>
>     - TSAN & ASAN. I don't think we can justify using 8 cores for
>     15-20 min for each of them.
>
>     - OpenCL, likely related to slow kernel compiles, which gets even
>     worse when the AMD GPUs become a bottleneck.
>
>
>     I also suspect that quite a few tests asking for lots of cores and
>     memory don't really use all of it (at least not efficiently), but
>     as a result other CI jobs will have to wait.
>
>     There's also a huge difference in performance between proper unit
>     tests called on code level vs. the ones that issue commands or
>     even run simulations.
>
>     This week is not the one to change things, but IMHO we need to get
>     back to the original model of the CI tests for every change
>     executing FAST. Any test job that doesn't complete in less than ~3
>     min on a single core does not belong among the ones that are run
>     for every change.
>
>     Cheers,
>
>     Erik
>
>
>
>
>     Erik Lindahl <erik.lindahl at scilifelab.se
>     <mailto:erik.lindahl at scilifelab.se>>
>     Professor of Biophysics
>     Science for Life Laboratory
>     Stockholm University & KTH
>     Office (SciLifeLab): +46 8 524 81567
>     Cell (Sweden): +46 73 4618050
>     Cell (US): +1 (650) 924 7674
>
>
>
>     > On 28 Sep 2020, at 16:23, Eric Irrgang <ericirrgang at gmail.com
>     <mailto:ericirrgang at gmail.com>> wrote:
>     >
>     > Hi Devs,
>     >
>     > If you push a new commit to a GitLab branch before the pipelines
>     are finished running for the previous commit, please consider
>     canceling one or the other sets of pipelines.
>     >
>     > You can look at the Pipelines tab of the merge request page (or
>     just go to https://gitlab.com/gromacs/gromacs/-/pipelines). If you
>     have pushed a new commit, you are presumably only interested in
>     one (of the sets of) pipelines. Just click the red X to cancel the
>     pipelines you don't need.
>     >
>     > If you are pushing to a branch that doesn't have an MR yet, you
>     are still generating one pipeline for every push, so please use
>     the web interface to cancel the pipelines that aren't useful to you.
>     >
>     > It will really help all of us to get our CI pipelines to run sooner.
>     >
>     > Thanks!
>     > M. Eric Irrgang
>     > --
>     > Gromacs Developers mailing list
>     >
>     > * Please search the archive at
>     http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>     before posting!
>     >
>     > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>     >
>     > * For (un)subscribe requests visit
>     >
>     https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>     or send a mail to gmx-developers-request at gromacs.org
>     <mailto:gmx-developers-request at gromacs.org>.
>     -- 
>     Gromacs Developers mailing list
>
>     * Please search the archive at
>     http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List
>     before posting!
>
>     * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
>     * For (un)subscribe requests visit
>     https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
>     or send a mail to gmx-developers-request at gromacs.org
>     <mailto:gmx-developers-request at gromacs.org>.
>

-- 
Paul Bauer, PhD
GROMACS Development Manager
KTH Stockholm, SciLifeLab
0046737308594

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20200928/5d4d6c2b/attachment-0001.html>