[gmx-users] tests for gmx 4.5

[Rossen Apostolov]

  Hi Mark,

It will be definitely important to have the regressiontests working 
properly and extended as the development of Gromacs continues. We 
discussed that with Berk, and here are some suggestions about the 
structure of the suite.

1. Make all current tests 3.3.4 compliant. There _is_ actually such a 
release :) but it hasn't been really announced.

I put a link to the tarball on the website, but also if you checkout the 
releases-3-3-patches branch ,the release is the last commit in that 
branch: d26be1f4015d974530e8a203e08ade01008e56a8. The bugs that have 
been fixed since 3.3.3 are now on the website: 
http://www.gromacs.org/About_Gromacs/Release_Notes/Revisions_in_versions_3.x. 


Are there other important issues with that code?

2. There is not much point in comparing the tpr's themselves. Not only 
it becomes problematic with introducing new options etc, but most of all 
the important point is to make sure that the simulations are reproducible.

So instead, check the development binaries (mdrun-dev, grompp-dev etc.) 
like that:

    * run mdrun-dev first with the 3.3.4-reference.tpr and compare
      against 3.3.4-reference.trr/.edr
    * run grompp-dev to generate a new dev.tpr and
    * run mdrun-dev with dev.tpr and compare again against
      3.3.4-reference.trr/.edr

3. The above will check only features present in 3.3.4 (and before). New 
.tpr files will be needed for the new features available in later 
releases. But those will fail to run with older mdrun.

But that's OK - if the error is simply "reading tpx file (topol.tpr) 
version 71 with version 40 program", then the test will be skipped and 
reported as "N/A" instead of "Failed". In that way the suite can be run 
against any version of mdrun.

It might be better to limit the number of steps, lets say 50, but have 
stricter checking of the error. With longer runs the trajectories will 
start deviating more anyway.

Berk modified yesterday the way gmxcheck compares the virials - now it 
uses only the diagonal elements of the tensor since the relative 
tolerance is very different between them and the off-diagonal ones.

Also, the .mdp files should be as small as possible. All default values 
should be stripped to keep the input easy to read. I already did that 
with all grompp4.mdp files in the complex/ and simple/ tests (and 
removed the grompp.mdp files), the kernel/* ones are already using very 
slim inputs. I put the changes in a new branch, origin/temp_mdp, to keep 
the master clean before we decide what to do.

4. pdb2gmx: There are new FFs in 4.5 that were not present in previous 
version which should be checked also. Similarly to the mdrun checks, if 
pdb2gmx returns errors like "Library file ffamber99.rtp not found in 
current dir nor in default directories." (in 4.0) or "Could not find 
force field 'amber98'" (in 4.5), then skip the test and mark it as 
"N/A". EncadFF should be removed, and Amber/CHARMM tests added maybe a 
little later when we're sure the inputs are working fine.


Finally, when all the tests pass with 4.5 we can incorporate them within 
CTest (there is a very basic support at the moment) and we'll have a 
nice testing system:)

Any comments are welcome.
Rossen



On 8/4/10 6:08 PM, Mark Abraham wrote:
> I'm glad there's some interest in the test set. I'm happy to update 
> what I did for 4.0.7 for 4.5 also, but there are a number of issues 
> that need attention from someone other than me. I listed those that 
> occurred to me here 
> http://lists.gromacs.org/pipermail/gmx-developers/2009-August/003586.html, 
> linked also from the Test Set page on the website.
>
> Among those issues is that we need a single GROMACS version that 
> produces reliable reference data for all cases we wish to test! 
> Various of the 3.3.x series have known issues that make test set 
> maintenance awkward.
>
> If I do the necessary updates to release-3-3-patches branch to make a 
> hypothetical GROMACS 3.3.4 that is useful for making test set 
> reference calculations, should we make such a release? Alternatively, 
> we can just announce that the reference data was generated by the 
> 3-3-version with a given git hash and not release. Thoughts?
>
> Mark
>
> ----- Original Message -----
> From: Rossen Apostolov <rossen.apostolov at cbr.su.se>
> Date: Wednesday, August 4, 2010 17:56
> Subject: Re: [gmx-users] tests for gmx 4.5
> To: gmx-users at gromacs.org
>
> > Hi,
> >
> > The tests haven't been ported to 4.5 yet but I hope to fix that 
> before the final release.
> >
> > Rossen
> >
> > On 7/31/10 7:08 PM, Alan wrote:
>
>> Is there a proper set of tests for gmx 4.5 because neither 
>> ftp://ftp.gromacs.org/pub/tests/gmxtest-4.0.4.tgz or "git clone 
>> git://git.gromacs.org/regressiontests.git 
>> <http://git.gromacs.org/regressiontests.git>" is working due to the 
>> several modifications done in gmx 4.5 including automatic thread for 
>> mdrun (I am using a Mac with dual core) and more warnings.
>>
>> > Thanks,
>> >
>> > Alan
>> >
>> > --
>> > Alan Wilter S. da Silva, D.Sc. - CCPN Research Associate
>> > Department of Biochemistry, University of Cambridge.
>> > 80 Tennis Court Road, Cambridge CB2 1GA, UK.
>> > >>http://www.bio.cam.ac.uk/~awd28 <http://www.bio.cam.ac.uk/%7Eawd28><<
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-users/attachments/20100805/9f93f8ad/attachment.html>