[gmx-developers] impending changes to Jenkins verification

[David van der Spoel]

On 02/09/15 23:12, Mark Abraham wrote:
> Hi devs,
>
> With 5.1 off the table, we're implementing some much-needed updates to
> the way we handle Jenkins verification of GROMACS.
>
> Teemu's rewritten the scripts we use to implement the various kinds of
> verification jobs, which will let us maintain and extend in much less
> ad-hoc fashion. Some parts of those scripts will now live in the GROMACS
> source repository, so that they can change in step with code changes.
> We've already submitted that script to master, so when you rebase
> patches over 0ce920a017, Jenkins will be able to use its new toys.

So is this the reason that most patches fail right now?
Where is this patch in gerrit? I can not seem to find it...

>
> We'll probably submit the half of the scripts that live in the releng
> repo shortly, and when we do, we'll make a new set of voting
> verification jobs named like the old ones, but with "-new-releng"
> suffix. These will all fail, unless a patch under review has been
> rebased to pick up the above commit. We plan to rebase a few commits to
> check the new gear is stable. Please feel free to rebase your commits
> and upload to Jenkins, but please check that Jenkins already doesn't
> have a hundred tasks in its queue, and don't rebase all your commits at
> once. Hopefully after a few days, we'll feel things are working well,
> and we will de-activate the old verification jobs and rely on the new ones.
>
> We then plan to rework the matrix of things that we get Jenkins to test
> every time code is uploaded to gerrit (ie. pre-submit), and use other
> matrices of build configurations and test types that run only after we
> submit code, or weekly, etc. This will cover a bunch of known test
> coverage gaps, and hopefully not too many unknown bugs will be found :-)
>
> At some point soon, we will probably move the matrix descriptions into
> the source repository, so we have a record of what build configurations
> and tests were supposed to pass on that code version. That might require
> another round of rebasing - you'll hear if that's the case.
>
> At some point soon, we plan to re-implement the build slaves that
> Jenkins uses, probably based on Docker containers running in a
> dynamically provisioned way on a bunch of old hardware we salvaged. This
> should give us reproducible testing behaviour. We should be able to
> spend less time on ongoing Jenkins maintenance. It should be easier for
> us to add a bunch of hardware during release periods for more rigorous
> testing. We can probably separate the compilation phase from the test
> phase (lowering the pressure on our machines with real target hardware
> in them). Debugging a problem reported in Jenkins will be easier if
> developers can get the Docker image that failed, and perhaps then run it
> on their desktop. Otherwise, hopefully we can extract the build and test
> artefacts from the Docker images, and publish them on Jenkins in the
> usual way.
>
> Any bright ideas, or other feedback is welcome. We'll keep you posted of
> developments :-)
>
> Mark
>
>


-- 
David van der Spoel, Ph.D., Professor of Biology
Dept. of Cell & Molec. Biol., Uppsala University.
Box 596, 75124 Uppsala, Sweden. Phone:	+46184714205.
spoel at xray.bmc.uu.se    http://folding.bmc.uu.se