[gmx-users] why does trjcat take so much memory?

[Roland Schulz]

On Wed, Oct 13, 2010 at 2:41 AM, David van der Spoel
<spoel at xray.bmc.uu.se>wrote:

> On 2010-10-13 01.54, chris.neale at utoronto.ca wrote:
>
>> I have 28,000 .xtc files, each having a single frame and each 150K. If I
>> run du -hs on the directory containing these .xtcs, I get 4.4 GB.
>> Nevertheless, when I run trjcat -f *.xtc -o ../tot.xtc , my memory
>> consumption goes over 11 GB and then I run out of available memory.
>>
> The program reads the first frame of all files, and since we roughly have
> three times compression from xtc to float your figures of 4 vs. 11 Gb. match
> perfect.
>
Each first frame is read twice. Thus I think it is 2x the memory.

>
> Although this is an extreme case, where there is a clear workaround (more
> memory) it could be worthwhile looking into an enhancement that releases the
> memory again after checking the first frame of all files. It is anyway
> impossible to keep 28,000 files open. Please file a bugzilla as an
> enhancement.
>
Yes and it should be easy because the frame isn't reused. Thus freeing it
shouldn't have any side effect.

Roland
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.sys.kth.se/pipermail/gromacs.org_gmx-users/attachments/20101013/627ee724/attachment.html>