[gmx-users] Running Grompp in reverse and backing up MD data
Mark Abraham
mark.j.abraham at gmail.com
Thu Nov 29 17:38:28 CET 2018
Hi,
Hopefully early in 2019 we will get to the point where the mdp inputs are
expressed as key-value data structures, perhaps in JSON. Once we do that,
that aspect of the .tpr no longer needs to be binary, and dumping it is
trivial.
Mark
On Thu, Nov 29, 2018 at 5:17 PM Thomas Piggot <t.piggot at soton.ac.uk> wrote:
> Good to know, thanks!
>
> Tom
>
> On 29/11/2018 16:08, Paul bauer wrote:
> > Hello,
> >
> > just a warning concerning the last message: The mdp from gmx dump -om
> > will not be the complete representation of your tpr, and you will not
> > be able to regenerate the tpr from this mdp file! I have been working
> > on a variant that would make this possible, but so far have not
> > managed it.
> >
> > Cheers
> > Paul
> >
> > On 29/11/2018 16:53, Thomas Piggot wrote:
> >> You can get an mdp from a tpr using gmx dump -om
> >>
> >> Cheers
> >>
> >> Tom
> >>
> >> On 29/11/2018 14:58, Peter Kroon wrote:
> >>> Hi all,
> >>>
> >>>
> >>> thanks for all the feedback.
> >>>
> >>> @Milan: I've made a tally of diskspace used per filetype in our lab. In
> >>> general people here don't save trr files at all (too expensive), yet
> >>> still out of 125TB total, 65% is made up of xtc files, and 10% is trr
> >>> files (followed by gro and xvg at 2.5% each). And that kind of backup
> >>> space becomes expensive. Of course for some simulations that were super
> >>> expensive to run we should back up the trajectories. But as a general
> >>> solution it's impossible. That also opens up the point of tiered
> >>> backups. For example: archive for the past 5 years to verify old
> >>> research results; off-site backups for the past 3 months in case the
> >>> building burns down; and local storage against accidental deletion. For
> >>> each there are different requirements and costs.
> >>>
> >>> @Mark: Part of the reason for this discussion is figuring out what my
> >>> responsibility is exactly. My thought was that as the TPR in general
> >>> contains the same information as the input used to make it, at least a
> >>> partial reverse should be possible. Then again, mdp/top/itp files are
> >>> pretty small (17GB/35GB/110GB respectively) and should be very
> >>> compressible. By comparison, tprs make up 880GB here.
> >>>
> >>> @Justin: Great point, thanks!
> >>>
> >>>
> >>> Thanks guys :)
> >>>
> >>>
> >>> Peter
> >>>
> >>>
> >>> On 29-11-18 15:29, Mark Abraham wrote:
> >>>> Hi,
> >>>>
> >>>> Yes, that's useful output, though strictly some mdp settings (like
> >>>> constraints and gen-vel) are consumed by grompp and thus not
> >>>> reported there.
> >>>>
> >>>> Mark
> >>>>
> >>>> On Thu, Nov 29, 2018 at 3:16 PM Justin Lemkul <jalemkul at vt.edu>
> wrote:
> >>>>
> >>>>> On 11/29/18 6:43 AM, Peter Kroon wrote:
> >>>>>> Hi Mark,
> >>>>>>
> >>>>>> thanks for the rapid reply!
> >>>>>>
> >>>>>> Hmmn. It doesn't have to be exactly matching input. An equivalent to
> >>>>>> e.g. mdout.mdp produced by grompp would already be good enough
> >>>>>> for my
> >>>>> All of the input .mdp settings are also written to the header of the
> >>>>> .log file for the run.
> >>>>>
> >>>>> -Justin
> >>>>>
> >>>>>> purposes. The same goes for a top file containing only the
> >>>>>> interactions
> >>>>>> relevant to the system.
> >>>>>>
> >>>>>>
> >>>>>> Peter
> >>>>>>
> >>>>>> On 29-11-18 11:31, Mark Abraham wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> Unfortunately there is no way to generate matching inputs to grompp
> >>>>> from a
> >>>>>>> .tpr file. In the longer term, we want that to be possible, but
> >>>>>>> it will
> >>>>>>> require a total rework of how users specify inputs, and that's a
> >>>>>>> large
> >>>>>>> effort!
> >>>>>>>
> >>>>>>> Mark
> >>>>>>>
> >>>>>>> On Thu, Nov 29, 2018 at 11:27 AM Peter Kroon <p.c.kroon at rug.nl>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> In the lab we're thinking about backup solutions for MD data
> >>>>>>>> (and in
> >>>>>>>> particular gromacs produced data). Since trajectories tend to
> >>>>>>>> be large
> >>>>>>>> we quickly decided that backing up the results of simulations is a
> >>>>> waste
> >>>>>>>> of disk space. Instead, we'll probably go for backing up the
> >>>>>>>> TPR files.
> >>>>>>>>
> >>>>>>>> Now for the question: Is there an easy way of generating the
> >>>>>>>> files used
> >>>>>>>> to generate the TPR files? For example the MDP file used to
> >>>>>>>> make a TPR
> >>>>>>>> file? How about a top file (I assume the original split between
> >>>>>>>> top and
> >>>>>>>> itp is lost)? I know I can get the starting configuration (gro)
> >>>>>>>> using
> >>>>>>>> editconf.
> >>>>>>>>
> >>>>>>>> And does anyone here have any other blinding insights on
> >>>>>>>> backing up MD
> >>>>>>>> data?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>>
> >>>>>>>> Peter
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Gromacs Users mailing list
> >>>>>>>>
> >>>>>>>> * Please search the archive at
> >>>>>>>>
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gromacs.org%2FSupport%2FMailing_Lists%2FGMX-Users_List&data=01%7C01%7Ct.piggot%40soton.ac.uk%7C84edd7637bf44b1aa1e008d65614ffe9%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&sdata=q47nQZj0RSrR%2BSVBkWVo5IZHAn7Fk8DVjLyWG7s5tKs%3D&reserved=0
> >>>>>>>> before
> >>>>>>>> posting!
> >>>>>>>>
> >>>>>>>> * Can't post? Read
> >>>>>>>>
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gromacs.org%2FSupport%2FMailing_Lists&data=01%7C01%7Ct.piggot%40soton.ac.uk%7C84edd7637bf44b1aa1e008d65614ffe9%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&sdata=064qx7R5%2F4wWz3CVmj7WP0a6J%2BZmVk3WCSMicgJgVL4%3D&reserved=0
> >>>>>>>>
> >>>>>>>> * For (un)subscribe requests visit
> >>>>>>>>
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmaillist.sys.kth.se%2Fmailman%2Flistinfo%2Fgromacs.org_gmx-users&data=01%7C01%7Ct.piggot%40soton.ac.uk%7C84edd7637bf44b1aa1e008d65614ffe9%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&sdata=v%2B%2BfxJx33%2FvHW7OAhvPLoCMjWczZ40LVl6nixk7QJTI%3D&reserved=0
> >>>>>>>> or
> >>>>>>>> send a mail to gmx-users-request at gromacs.org.
> >>>>>>>>
> >>>>> --
> >>>>> ==================================================
> >>>>>
> >>>>> Justin A. Lemkul, Ph.D.
> >>>>> Assistant Professor
> >>>>> Office: 301 Fralin Hall
> >>>>> Lab: 303 Engel Hall
> >>>>>
> >>>>> Virginia Tech Department of Biochemistry
> >>>>> 340 West Campus Dr.
> >>>>> Blacksburg, VA 24061
> >>>>>
> >>>>> jalemkul at vt.edu | (540) 231-3129
> >>>>>
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.thelemkullab.com&data=01%7C01%7Ct.piggot%40soton.ac.uk%7C84edd7637bf44b1aa1e008d65614ffe9%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&sdata=TVKz0Drugreb7SS3lt%2B9J%2FEUJTWOo8o77n88eyOO%2BQY%3D&reserved=0
> >>>>>
> >>>>>
> >>>>> ==================================================
> >>>>>
> >>>>> --
> >>>>> Gromacs Users mailing list
> >>>>>
> >>>>> * Please search the archive at
> >>>>>
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gromacs.org%2FSupport%2FMailing_Lists%2FGMX-Users_List&data=01%7C01%7Ct.piggot%40soton.ac.uk%7C84edd7637bf44b1aa1e008d65614ffe9%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&sdata=q47nQZj0RSrR%2BSVBkWVo5IZHAn7Fk8DVjLyWG7s5tKs%3D&reserved=0
> >>>>> before
> >>>>> posting!
> >>>>>
> >>>>> * Can't post? Read
> >>>>>
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gromacs.org%2FSupport%2FMailing_Lists&data=01%7C01%7Ct.piggot%40soton.ac.uk%7C84edd7637bf44b1aa1e008d65614ffe9%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&sdata=064qx7R5%2F4wWz3CVmj7WP0a6J%2BZmVk3WCSMicgJgVL4%3D&reserved=0
> >>>>>
> >>>>> * For (un)subscribe requests visit
> >>>>>
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmaillist.sys.kth.se%2Fmailman%2Flistinfo%2Fgromacs.org_gmx-users&data=01%7C01%7Ct.piggot%40soton.ac.uk%7C84edd7637bf44b1aa1e008d65614ffe9%7C4a5378f929f44d3ebe89669d03ada9d8%7C1&sdata=v%2B%2BfxJx33%2FvHW7OAhvPLoCMjWczZ40LVl6nixk7QJTI%3D&reserved=0
> >>>>> or
> >>>>> send a mail to gmx-users-request at gromacs.org.
> >>>>>
> >>
> >
>
> --
> Dr Thomas Piggot
> Visiting Fellow
> University of Southampton, UK.
>
> --
> Gromacs Users mailing list
>
> * Please search the archive at
> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before
> posting!
>
> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
>
> * For (un)subscribe requests visit
> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or
> send a mail to gmx-users-request at gromacs.org.
>
More information about the gromacs.org_gmx-users
mailing list