[gmx-developers] could somebody fix the regressiontests package?

[Szilárd Páll]

On Wed, Feb 27, 2013 at 6:51 PM, Mark Abraham <mark.j.abraham at gmail.com>wrote:

>
>
> On Tue, Feb 26, 2013 at 9:31 PM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:
>
>> On Tue, Feb 26, 2013 at 5:50 PM, Roland Schulz <roland at utk.edu> wrote:
>>
>>>
>>>
>>>
>>> On Tue, Feb 26, 2013 at 10:25 AM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:
>>>
>>>>  Bump!
>>>>
>>>>  I have not had time to figure out a solution. Does anybody have
>>>> suggestions on how to fix this?
>>>>
>>> you simply upload a fixed version. I think the proper fix is that for
>>> the git version, git is used to download. It only makes sense to download a
>>> TGZ from http if the source is also a tar ball. But I was waiting for us to
>>> decide whether we will change regressiontests to a submodule before I
>>> change that.
>>>
>>
>> I see. I would still like to have it fixed asap and would have done it
>> myself, but Mark suggested that one can not simply replace a file.
>>
>>
>>>
>>>
>>>> I have a serious concern: as nobody has noticed this except me, I think
>>>> people are simply running the regressiontests (while I do know that many
>>>> people are using the source from git). I tend to think this is a problem of
>>>> communication/documentation and we should try to improve on this aspect.
>>>> Not only that if the tests are not used it was a half-futile effort to make
>>>> them work, but knowing how fragile intrinsic-based kernels are, it is also
>>>> dangerous.
>>>>
>>> According to https://code.google.com/p/gromacs/downloads/list the 4.6
>>> was downloaded 328. I think this makes this useful. master was only
>>> downloaded 15 so it seems that my assumption that it is mostly useful to
>>> people with tar ball not git is mostly true.
>>>
>>
>> In general, I don't think we should jump to conclusions just yet because:
>>
>
> I'm not sure what the conclusion is to which you might be objecting. Users
> have downloaded the tests, which is a good thing. We haven't had a forest
> of complaints about it, but we can't know whether that's a good or bad
> thing.
>

This one: "seems that my assumption that it is mostly useful to people with
tar ball not git is mostly true".

AFAIK most advanced users would expect to get a tarball or git code with
self-contained README/INSTALL files that provide brief information on,
among other things, compilation. As the INSTALL file in the release was not
updated, it doesn't mention "make check" & related things.


>
>
>> - The feature has just been introduced;
>> - I have the feeling that validating a GROMACS build has not been
>> encouraged enough:
>>   - I just realized hat even the release notes do not mention this new
>> feature (added).
>>   - The INSTALL file does not mention it;
>>   - It might be a good idea to write a separate email about it to the
>> mailing list (will do it, but I'd prefer to have the issue fixed first);
>> - As the auto-download with git source has been buggy some might have
>> simply switched to simply checking out the regressiontests repo exactly
>> because of the bug.
>>
>
> Contributions of time are welcome. I'm paid to manage the development of
> GROMACS, but if I fixed every little blemish any of us can see, then 5.0
> will never happen. :-) If there's a reasonable way for users to find out
> that we have some tests (e.g. the install guide on the web) and to run them
> fairly easily (which they can do two different ways) then I think we've
> done well.
>

While I've been using *NIX and OSS software not much longer than a decade,
AFAIK README, INSTALL, and CHAGELOG files are the standard. Feel free to
correct me on this if I'm wrong.


>
> While it is probably true that the tarballs are more useful for users
>> downloading the source, I do think that it there are many advanced users
>> using the git code-base who would benefit from it. I can only guess that
>> these more advanced users are probably less likely to read through the
>> install guide or simply be more confident that they'd notice if something
>> goes wrong because of mis-compilation or other platform-specific bugs.
>> Hence, although the number of users using git code is probably much smaller
>> than source-tarball users, I would guess that these users are more likely
>> to simply not be aware of the integrated regression-tests.
>>
>
> I don't follow you. If any kind of user won't scan the table of contents
> of an install guide on the web, and thereby manages to remain ignorant that
> we have some automated testing, that's a problem to which they've
> contributed.
>

I disagree. Se above


> If we offer an automated tarball download, then we should make it work
> correctly. I have replaced the old tarball with one that works. I've also
> built and uploaded a 4.6.1 test tarball, and updated the machinery by which
> we do this for future patch releases.
>

Great. I'll put together an email after 4.6.1 which briefly explains what
is regression testing and why should everybody do it.

Cheers,
Szilard

>
> Mark
>
> --
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-developers-request at gromacs.org.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20130227/aae14ca0/attachment.html>