[gmx-developers] New Test Set

[Berk Hess]

Hi,

I agree test sets are very important.
Having good tests will make development and especially the process of 
accepting contributions much easier.

Now that we have the new, by default, energy conserving loops, I realize 
that energy conservation
is extremely useful for validation. I think that having tests that check 
energy conservation and particular
energy values of particular (combinations of) functionality will catch a 
lot of problems.
The problems is that MD is chaotic and with non energy-conserving setups 
the divergence is extremely fast.
With energy conservation running 20 steps with nstlist=10, checking the 
conserved energy + a few terms
would be enough for testing most modules, I think.
We still want some more extended tests, but that could be a separate set.

So setting up a framework for the simple tests should not be too hard.
Then we need to come up with a set of tests and reference values.

Cheers,

Berk

On 02/05/2012 04:56 AM, Roland Schulz wrote:
> Hi,
>
> we agreed that we would want to have a test set for 4.6 but so far we 
> haven't made any progress on it (as far as I know). I want to try to 
> get this work started by posting here a list of questions I have about 
> the new test set. Please add your own questions and answer any 
> questions you can (no need to try to answer all questions).
>
> - Why do the current tests fail? Is it only because of different 
> floating point rounding or are there other problems? What's the best 
> procedure to find out why a test fails?
> - Which tests should be part of the new test set?
> - Should the current tests all be part of the new test set?
> - How should the new test be implemented? Should the comparison with 
> the reference value be done in C (within mdrun), ctest script, python 
> or perl?
> - Should the new test execute mdrun for each test? Or should we 
> somehow (e.g. python wrapper or within mdrun) load the binary only 
> once and run many test per execution?
> - What are the requirements for the new test set? E.g. how easy should 
> it be to see whats wrong when a test fails? Should the test support 
> being run under valgrind? Other?
> - Do we have any other bugs which have to be solved before the test 
> can be implemented? E.g. is the problem with shared libraries solved? 
> Are there any open redmine issues related to the new test set?
> - Should we have a policy that everyone who adds a feature also has to 
> provide tests covering those features?
> - Should we have a conference call to discuss the test set? If yes when?
> - Should we agree that we won't release 4.6 without the test set to 
> give it a high priority?
> Roland
>
> -- 
> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov <http://cmb.ornl.gov>
> 865-241-1537, ORNL PO BOX 2008 MS6309
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20120206/9c3688b2/attachment.html>