[gmx-developers] automated performance testing

[Mark Abraham]

On Tue, Sep 30, 2014 at 6:48 PM, Roland Schulz <roland at utk.edu> wrote:

>
>
> On Tue, Sep 30, 2014 at 12:22 PM, Mark Abraham <mark.j.abraham at gmail.com>
> wrote:
>
>>   Hi,
>>
>>  Cherry-picking Michael's email into its own thread:
>>
>>  > Is there a plan (long term) to do (essentially) automated performance
>> tests so that we
>> > can perform consistent(ish) checks for new changes in code, then post
>> the results in an
>> > easy(ish) way to interpret for others?
>>
>>  There's no organized plan. I've lately been trying to organize a
>> dedicated machine here so we can start to do some of this - had we had it
>> and the right kinds of tests then various bugs would not have gone
>> unnoticed. In principle, people could run that test suite on their own
>> hardware, of course.
>>
>>  One option I've been toying with lately is dumping the mdrun
>> performance data matrix to XML (or something) so that some existing
>> plotting machinery can show the trend over time (and also observe a
>> per-commit delta large enough to vote -1 on Jenkins). I also mean to have a
>> poke around with http://www.phoromatic.com/ to see if maybe it already
>> has infrastructure we could use.
>>
>
> Vtune provides a function for that:
> https://software.intel.com/en-us/node/529460
>

Interesting, that might prove useful.

Yesterday, I learned of JuBE
http://www.fz-juelich.de/ias/jsc/EN/Expertise/Support/Software/JUBE/_node.html,
which seems like a fairly fully-featured benchmarking infrastructure suite.
You can pre-organize pretty much anything, and using it might save us from
implementing wheels again.

Even somewhat better, other people at the CRESTA project have taken a small
4.5.5 benchmark set we gave them and deployed it in JuBE (along with some
other CRESTA applications). That needs some updating to work with the new
gear, which I am coordinating with them. It's currently only available
internally (licensing questions with other apps), but the public release
will be in the coming weeks. We'd be fine to take the Gromacs part, and
adapt for whichever purposes we want.

Mark

Roland
>
>>
>>  Mark
>>
>
>
>
> --
> ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
> 865-241-1537, ORNL PO BOX 2008 MS6309
>
> --
> Gromacs Developers mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-developers_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-developers
> or send a mail to gmx-developers-request at gromacs.org.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-developers/attachments/20141022/a7cae508/attachment.html>