[gmx-users] why does trjcat take so much memory?

[Berk Hess]

Hi,

You can simply use plain unix cat, that will be much faster and does not use memory.
If you want the time of the frames right, run trjconv with the right option on the catted
frames.

Berk

> Date: Wed, 13 Oct 2010 08:41:17 +0200
> From: spoel at xray.bmc.uu.se
> To: gmx-users at gromacs.org
> Subject: Re: [gmx-users] why does trjcat take so much memory?
> 
> On 2010-10-13 01.54, chris.neale at utoronto.ca wrote:
> > I have 28,000 .xtc files, each having a single frame and each 150K. If I
> > run du -hs on the directory containing these .xtcs, I get 4.4 GB.
> > Nevertheless, when I run trjcat -f *.xtc -o ../tot.xtc , my memory
> > consumption goes over 11 GB and then I run out of available memory.
> The program reads the first frame of all files, and since we roughly 
> have three times compression from xtc to float your figures of 4 vs. 11 
> Gb. match perfect.
> 
> Although this is an extreme case, where there is a clear workaround 
> (more memory) it could be worthwhile looking into an enhancement that 
> releases the memory again after checking the first frame of all files. 
> It is anyway impossible to keep 28,000 files open. Please file a 
> bugzilla as an enhancement.
> 
> Another workaround is to do this in 10 steps or so. Some scripting 
> required but not too bad.
> 
> >
> > Sure, I could find a larger memory system, but why does it take >=3x
> > more memory than the individual files took in disk space? Does anybody
> > know any commands
> >
> > This is gromacs 4.0.5 and 4.0.7 and 4.5-beta2.
> >
> > Thank you,
> > Chris.
> >
> 
> 
> -- 
> David van der Spoel, Ph.D., Professor of Biology
> Dept. of Cell & Molec. Biol., Uppsala University.
> Box 596, 75124 Uppsala, Sweden. Phone:	+46184714205.
> spoel at xray.bmc.uu.se    http://folding.bmc.uu.se
> -- 
> gmx-users mailing list    gmx-users at gromacs.org
> http://lists.gromacs.org/mailman/listinfo/gmx-users
> Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/Search before posting!
> Please don't post (un)subscribe requests to the list. Use the 
> www interface or send it to gmx-users-request at gromacs.org.
> Can't post? Read http://www.gromacs.org/Support/Mailing_Lists
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-users/attachments/20101013/d560afe5/attachment.html>