[gmx-developers] More regression test issues

[Mark Abraham]

Hi,

1) It seems many of the reference runs in the original regression test 
module were generated with a pre-release version of 3.3.2, rather than 
3.3.2 as suggested by the README. This means some minor differences 
occur in the .tpr and mdout.mdp files, so I'm regenerating all of these.

2) simple/rb1 segfaulted in mdrun on my 3.3.2 installation and I could 
see that it had "define = -DRYCKAERT -DMOL1" where simple/rb125 had 
"-DRYCKAERT -DBONDS", and moreover MOL1 was never used. When I replaced 
MOL1 with BONDS, the run appeared to work fine. Can whoever designed 
this test (DvdS?) comment, please?

3) In complex tests, comparison of 4.x versions with 3.3.x reference 
.edr files does not succeed because the number of terms in the energy 
files can change (e.g. introduction of  "Cons.-rmsd-()"). gmxcheck 
reports this mismatch, but the original gmxtest.pl did not flag this 
case. Presumably since it was written before 4.x, this problem could not 
arise. Anyway, subsequent virial checks appear to pass, but no 
comparison actually occurs! Accordingly, we either need a robust way to 
strip 4.x .edr files back to match 3.3.x reference .edr files, or to 
supply both 3.3.x and 4.x reference files for comparison. Assuming the 
latter is the most practical course, I plan to use 4.0.5 for the 
reference version. Comments?

Also, gmxtest.pl needs to check return values of calls to utilities for 
success, rather than assuming it, and I'll implement that.

4) Also when testing 4.x, the mdout.mdp files will differ greatly from 
the input .mdp files (which are outputs from 3.3.x grompp). Rather than 
pass silently, or warn the user of a non-problem, I propose to check 
against the grompp output of a suitable 4.x version. This combines with 
my suggested solution to 3) above.

5) Since the Buckingham kernel test references need to be generated with 
4.x, they will have the converse problem of 4), as well as certain 
failure when testing 3.3.x versions. I've added a note to the output to 
tell the user that this is a known bug.

6) What's the best way to me to get my fixed version of this package 
into the git tree for distribution and further testing?

Mark