Added by Szilárd Páll about 1 year ago. Updated 8 months ago.
OpenCL runs suffer from a memory leak which after a few hours of runtime will consume all host memory. It seems to be related to GPU timing as the leak is not present if timing is turned off. Likely surfaced as a result of #2468.
Actually, even non-DD runs do leak memory, so there is something strange going on here. Disabling timing seems to strongly reduce or eliminate the effect; there is still a strange drift in memory usage, but it seems to plateau after a while. Need longer runs to assess it.
Gerrit received a related DRAFT patchset '2' for Issue #2470.
Let me repeat my question from gerrit: have you verified that long runs do not show any indication of leaks?
Log files with output of free kbytes are attached.
The version without GpuRegionTimer I actually ran much longer, so it has a flat part in the end (after gmx stopped).
The trouble with the 4th column of free is that is the true "free" memory, not including disc cache, etc. For that reason, as mdrun does write at least to the log, checkpoint, and possibly energy output, that is not a good metric.
I suggest looking at the "used" column of free and/or the resident memory usage of the gmx process.