[gmx-developers] lots of new SIMD code but very poor jenkins coverage!

[Mark Abraham]

Hi,

On Thu, Mar 10, 2016 at 4:56 PM Szilárd Páll <pall.szilard at gmail.com> wrote:

> On Wed, Mar 9, 2016 at 4:43 PM, Mark Abraham <mark.j.abraham at gmail.com>
> wrote:
>
>> Hi,
>>
>>
>> On Wed, Mar 9, 2016 at 3:42 PM Szilárd Páll <pall.szilard at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> No, I don't. I triggered it the other day because I did not know the
>>> main 5.1 matrix does the job already.
>>>
>>
>> OK, I'll delete it tomorrow.
>>
>>
>>> What about master, additional releng support is needed I guess? Also, is
>>> there anything holding us back from adding pre- and post-submit configs
>>> to admin//builds/*matrix.txt.
>>>
>>
>> See my other email to gmx-developers just now. I think pre-submit should
>> cover only the highest-value targets. Post-submit for lower-value targets
>> and/or tests that take more time.
>>
>
> I agree. A dozen or so configs to cover the most common
> software/parallelization configurations should be enough. Having looked at
> the Gromacs_Gerrit_master_nrwpo configs,
>

Don't, that's just an example set so people can run a custom config if they
have a need to do so. I have now clarified that in the Jenkins parameter
description text. Per my other email, the matrix lives in the source repo
at admin/builds/pre-submit-matrix.txt

I propose the following additions/tweaks:
> - OpenCL (I suggest using AMD GPUs and adding both release and debug,
> error handling is rather poor and OpenCL API erros are mostly handled with
> assert's)
>

Sure, but first we have to make the build work (there were mysterious
failures last time I tried it).


> - an SSE4.1 (i.e. 4-wide SIMD) build unless  - AFAICT VM-based builds are
> not already doing that;
>

Plausible, but why is it a "highest-value target?" That and SSE2 seem like
candidates for post-submit testing. If some patch fails to break other
kinds of SIMD, it's pretty unlikely to break precisely these, unless the
patch is somehow aimed at them.

- some runs that execute tests with 2D/3D DD (if 3D is hw resource-wise an
> issue could be left for post-submit),
>

I don't remember whether this still happens automagically. Running some
multi-rank tests seems wise, but yeah 3D can probably take a lower priority
if 2D is covered.

Additionally:
> - Has valgrind testing been discontinued? (Teemu mentioned on #1815 that
> valgrind configs got accidentally removed at some point?)
>

AFAICR it's currently disabled with at least Windows, icc, gcc-5 and clang
builds, for issues including those caused by

http://valgrind.org/docs/manual/faq.html#faq.reports
https://gcc.gnu.org/onlinedocs/libstdc++/manual/debug.html#debug.memory

If we want to use it in Jenkins in future, then I think we need to commit
to running it with a C++ standard library in "always free memory mode,"
which is going to be even slower to run and/or harder to maintain on "real"
build slaves. Valgrind is approximately superseded by MemorySanitizer, for
which we have a build type and a probably-not-badly-broken build at
http://jenkins.gromacs.org/job/Gromacs_Gerrit_master-test-MSan/. These
kinds of tools are particularly brittle with respect to dependencies, etc.
so I think they make sense to run regularly in Jenkins only if we implement
them in a Docker environment, built with a script (Google suggests some
people might have done this for us already), rather than on some build
slave whose setup details nobody can remember and will break upon upgrades
:-)

- How about TSAN? I've seen no confgis use it (but we've been using it for
> 5.1, right?)
>

It's in the actual matrix already. In the longer term, I think it also
makes sense on a Docker image.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20160310/f6b95299/attachment-0001.html>