[gmx-developers] New Test Set

[Roland Schulz]

On Mon, Feb 6, 2012 at 10:17 AM, Berk Hess <hess at kth.se> wrote:

>  Hi,
>
> I agree test sets are very important.
> Having good tests will make development and especially the process of
> accepting contributions much easier.
>
> Now that we have the new, by default, energy conserving loops, I realize
> that energy conservation
> is extremely useful for validation. I think that having tests that check
> energy conservation and particular
> energy values of particular (combinations of) functionality will catch a
> lot of problems.
> The problems is that MD is chaotic and with non energy-conserving setups
> the divergence is extremely fast.
> With energy conservation running 20 steps with nstlist=10, checking the
> conserved energy + a few terms
> would be enough for testing most modules, I think.
> We still want some more extended tests, but that could be a separate set.
>
> So setting up a framework for the simple tests should not be too hard.
>
What tests do you have in mind for these simple tests? Would these be
all integration tests?

Roland


>
> On 02/05/2012 04:56 AM, Roland Schulz wrote:
>
> Hi,
>
>  we agreed that we would want to have a test set for 4.6 but so far we
> haven't made any progress on it (as far as I know). I want to try to get
> this work started by posting here a list of questions I have about the new
> test set. Please add your own questions and answer any questions you can
> (no need to try to answer all questions).
>
>  - Why do the current tests fail? Is it only because of different
> floating point rounding or are there other problems? What's the best
> procedure to find out why a test fails?
> - Which tests should be part of the new test set?
> - Should the current tests all be part of the new test set?
> - How should the new test be implemented? Should the comparison with the
> reference value be done in C (within mdrun), ctest script, python or perl?
> - Should the new test execute mdrun for each test? Or should we somehow
> (e.g. python wrapper or within mdrun) load the binary only once and run
> many test per execution?
> - What are the requirements for the new test set? E.g. how easy should it
> be to see whats wrong when a test fails? Should the test support being run
> under valgrind? Other?
> - Do we have any other bugs which have to be solved before the test can be
> implemented? E.g. is the problem with shared libraries solved? Are there
> any open redmine issues related to the new test set?
> - Should we have a policy that everyone who adds a feature also has to
> provide tests covering those features?
> - Should we have a conference call to discuss the test set? If yes when?
> - Should we agree that we won't release 4.6 without the test set to give
> it a high priority?
> Roland
>
>  --
> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> 865-241-1537, ORNL PO BOX 2008 MS6309
>
>
>
>


-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20120210/d190ee8a/attachment.html>