[gmx-developers] could somebody fix the regressiontests package?

Wed Feb 27 20:19:23 CET 2013

On Wed, Feb 27, 2013 at 7:53 PM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:

>
> On Wed, Feb 27, 2013 at 6:51 PM, Mark Abraham <mark.j.abraham at gmail.com>wrote:
>
>>
>>
>> On Tue, Feb 26, 2013 at 9:31 PM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:
>>
>>> On Tue, Feb 26, 2013 at 5:50 PM, Roland Schulz <roland at utk.edu> wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Tue, Feb 26, 2013 at 10:25 AM, Szilárd Páll <szilard.pall at cbr.su.se>wrote:
>>>>
>>>>>  Bump!
>>>>>
>>>>>  I have not had time to figure out a solution. Does anybody have
>>>>> suggestions on how to fix this?
>>>>>
>>>> you simply upload a fixed version. I think the proper fix is that for
>>>> the git version, git is used to download. It only makes sense to download a
>>>> TGZ from http if the source is also a tar ball. But I was waiting for us to
>>>> decide whether we will change regressiontests to a submodule before I
>>>> change that.
>>>>
>>>
>>> I see. I would still like to have it fixed asap and would have done it
>>> myself, but Mark suggested that one can not simply replace a file.
>>>
>>>
>>>>
>>>>
>>>>> I have a serious concern: as nobody has noticed this except me, I
>>>>> think people are simply running the regressiontests (while I do know that
>>>>> many people are using the source from git). I tend to think this is a
>>>>> problem of communication/documentation and we should try to improve on this
>>>>> aspect. Not only that if the tests are not used it was a half-futile effort
>>>>> to make them work, but knowing how fragile intrinsic-based kernels are, it
>>>>> is also dangerous.
>>>>>
>>>> According to https://code.google.com/p/gromacs/downloads/list the 4.6
>>>> was downloaded 328. I think this makes this useful. master was only
>>>> downloaded 15 so it seems that my assumption that it is mostly useful to
>>>> people with tar ball not git is mostly true.
>>>>
>>>
>>> In general, I don't think we should jump to conclusions just yet because:
>>>
>>
>> I'm not sure what the conclusion is to which you might be objecting.
>> Users have downloaded the tests, which is a good thing. We haven't had a
>> forest of complaints about it, but we can't know whether that's a good or
>> bad thing.
>>
>
> This one: "seems that my assumption that it is mostly useful to people
> with tar ball not git is mostly true".
>

People with the gromacs 4.6 tarball got the correctly-working
regressiontests-4.6.tar.gz around 328 times. People with the post-4.6 git
branch tried to get the broken regressiontests-master.tar.gz many fewer
times. We don't know how many people with either did no testing. Shrug.

AFAIK most advanced users would expect to get a tarball or git code with
> self-contained README/INSTALL files that provide brief information on,
> among other things, compilation. As the INSTALL file in the release was not
> updated, it doesn't mention "make check" & related things.
>

Sure. But AFAIK nobody else has complained that such files missing, either.

> - The feature has just been introduced;
>>> - I have the feeling that validating a GROMACS build has not been
>>> encouraged enough:
>>>   - I just realized hat even the release notes do not mention this new
>>> feature (added).
>>>   - The INSTALL file does not mention it;
>>>   - It might be a good idea to write a separate email about it to the
>>> mailing list (will do it, but I'd prefer to have the issue fixed first);
>>> - As the auto-download with git source has been buggy some might have
>>> simply switched to simply checking out the regressiontests repo exactly
>>> because of the bug.
>>>
>>
>> Contributions of time are welcome. I'm paid to manage the development of
>> GROMACS, but if I fixed every little blemish any of us can see, then 5.0
>> will never happen. :-) If there's a reasonable way for users to find out
>> that we have some tests (e.g. the install guide on the web) and to run them
>> fairly easily (which they can do two different ways) then I think we've
>> done well.
>>
>
> While I've been using *NIX and OSS software not much longer than a decade,
> AFAIK README, INSTALL, and CHAGELOG files are the standard. Feel free to
> correct me on this if I'm wrong.
>

That is a standard thing. It's particularly important for code that is
often a dependency for other code, because you want to be able to compile
and use something correct ASAP. It's less important for code where there
are few things that depend on them, and where effective compilation is a
non-trivial process. If the user is likely going to have to stop and do
some research in order to build GROMACS well, making them go to the webpage
to get access to any instructions might even work out better in the long
run.

More to the point, I'm not going to spend time duplicating such content,
particularly if it has to get bumped for roughly monthly patch releases (or
else go out of date), and so go through code review processes that are
mostly provided by busy volunteers. That ensures we can't have timely
releases. If there's a way to deliver such text files for free such that
they're only maintained once, then that's good. But so far there isn't such
a way.

Mark

While it is probably true that the tarballs are more useful for users
>>> downloading the source, I do think that it there are many advanced users
>>> using the git code-base who would benefit from it. I can only guess that
>>> these more advanced users are probably less likely to read through the
>>> install guide or simply be more confident that they'd notice if something
>>> goes wrong because of mis-compilation or other platform-specific bugs.
>>> Hence, although the number of users using git code is probably much smaller
>>> than source-tarball users, I would guess that these users are more likely
>>> to simply not be aware of the integrated regression-tests.
>>>
>>
>> I don't follow you. If any kind of user won't scan the table of contents
>> of an install guide on the web, and thereby manages to remain ignorant that
>> we have some automated testing, that's a problem to which they've
>> contributed.
>>
>
> I disagree. Se above
>
>
>> If we offer an automated tarball download, then we should make it work
>> correctly. I have replaced the old tarball with one that works. I've also
>> built and uploaded a 4.6.1 test tarball, and updated the machinery by which
>> we do this for future patch releases.
>>
>
> Great. I'll put together an email after 4.6.1 which briefly explains what
> is regression testing and why should everybody do it.
>
> Cheers,
> Szilard
>
>>
>> Mark
>>
>> --
>> gmx-developers mailing list
>> gmx-developers at gromacs.org
>> http://lists.gromacs.org/mailman/listinfo/gmx-developers
>> Please don't post (un)subscribe requests to the list. Use the
>> www interface or send it to gmx-developers-request at gromacs.org.
>>
>
>
> --
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-developers-request at gromacs.org.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20130227/d6fa7367/attachment.html>