[gmx-developers] New Test Set

[Shirts, Michael (mrs5pt)]

Hi, all-

> Personally, I think that the most important tests are those that
> validate the code. See, for example, the tip4p free-energy code bug:
> 
> http://www.mail-archive.com/gmx-users@gromacs.org/msg18846.html
> 
> I think that it is this type of error, which can silently lead to the
> wrong values, that we really need a test set in order to catch. The
> gmx-users list is not such a bad way to catch errors that lead to
> grompp failure.
> 
> It is not immediately clear, however, who would be responsible for
> developing such a test (those who added tip4p, those who added
> optimization loops, or those who added the free energy code). I recall
> seeing a post indicating that developers would be required to test
> their code prior to incorporation, but with so many usage options in
> mdrun I think that it will be essential to figure out how to define
> that requirement more precisely.

I agree that these tests are much harder to catch, and these are the sorts
of tests that can be developed on a longer time scale.  I strongly suspect
that some of these bugs will actually cause run failures for some
combination of input parameters, so a wide enough set of "does it run" tests
will catch them -- if it's doing something wrong, there is a decent chance
it will cause a catastrophic failure for some set of options.

Other bugs will be almost impossible to catch looking at Gromacs alone,
because they will do things that are physically consistent, but doing
something other than one thinks they should be doing. I'm in discussions
with some other chemical engineers about how to perform this type of
validation -- basically, the best bet is to have databases of results of
particular molecular models obtained on lots of different softwares.  NIST
has some interest in this (for example, they have a database of
Lennard-Jonesium results), so hopefully something comes out of this sort of
collaboration in the next few years.    Obviously, not something that can be
prepared for 4.6.

Best,
~~~~~~~~~~~~
Michael Shirts
Assistant Professor
Department of Chemical Engineering
University of Virginia
michael.shirts at virginia.edu
(434)-243-1821