[gmx-developers] More regression test issues

[hess at sbc.su.se]

> Hi,
>
> 1) It seems many of the reference runs in the original regression test
> module were generated with a pre-release version of 3.3.2, rather than
> 3.3.2 as suggested by the README. This means some minor differences
> occur in the .tpr and mdout.mdp files, so I'm regenerating all of these.
>
> 2) simple/rb1 segfaulted in mdrun on my 3.3.2 installation and I could
> see that it had "define = -DRYCKAERT -DMOL1" where simple/rb125 had
> "-DRYCKAERT -DBONDS", and moreover MOL1 was never used. When I replaced
> MOL1 with BONDS, the run appeared to work fine. Can whoever designed
> this test (DvdS?) comment, please?
>
> 3) In complex tests, comparison of 4.x versions with 3.3.x reference
> .edr files does not succeed because the number of terms in the energy
> files can change (e.g. introduction of  "Cons.-rmsd-()"). gmxcheck
> reports this mismatch, but the original gmxtest.pl did not flag this
> case. Presumably since it was written before 4.x, this problem could not
> arise. Anyway, subsequent virial checks appear to pass, but no
> comparison actually occurs! Accordingly, we either need a robust way to
> strip 4.x .edr files back to match 3.3.x reference .edr files, or to
> supply both 3.3.x and 4.x reference files for comparison. Assuming the
> latter is the most practical course, I plan to use 4.0.5 for the
> reference version. Comments?
>

gmxcheck -lastener allows terms to be ignored.
We could ignore starting from constr-rmsd.
That would skip the virial terms (as is maybe done now already),
but if we compare pressure we indirectly compare the virial.

Another option is to build name searching into gmxcheck
such that it properly matches up the names.

Another option is to forget about 3.? checks altogher for things
that have changed and only provide reference for 4.0.

Checks currently seem to be with rather large margins.
It would be nice if we could make them stricter,
but we have to be extremely careful that architecture,
compiler etc do not cause to large differences.

> Also, gmxtest.pl needs to check return values of calls to utilities for
> success, rather than assuming it, and I'll implement that.
>
> 4) Also when testing 4.x, the mdout.mdp files will differ greatly from
> the input .mdp files (which are outputs from 3.3.x grompp). Rather than
> pass silently, or warn the user of a non-problem, I propose to check
> against the grompp output of a suitable 4.x version. This combines with
> my suggested solution to 3) above.
>
> 5) Since the Buckingham kernel test references need to be generated with
> 4.x, they will have the converse problem of 4), as well as certain
> failure when testing 3.3.x versions. I've added a note to the output to
> tell the user that this is a known bug.
>
> 6) What's the best way to me to get my fixed version of this package
> into the git tree for distribution and further testing?

Give you git access?

Berk


>
> Mark
> _______________________________________________
> gmx-developers mailing list
> gmx-developers at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-developers
> Please don't post (un)subscribe requests to the list. Use the
> www interface or send it to gmx-developers-request at gromacs.org.
>