GPU memory leakage when creating objects from sentence-transformers
Description
I am creating a function in R that embeds sentences using the sentence_transformers library from Python.
For some unknown reason, creating the object multiple times under the same variable name ends up in insufficient memory space to allocate the transformer. To reproduce:
sentence_transformers &lt;- reticulate::import(&quot;sentence_transformers&quot;)
for (i in 1:10) {
  print(i)
  bert_encoder &lt;- sentence_transformers$SentenceTransformer(&quot;bert-large-nli-stsb-mean-tokens&quot;)
}

However, doing the same operation directly on Python does not produce an error
from sentence_transformers import SentenceTransformer
for i in range(10):
    print(i)
    bert_encoder = SentenceTransformer(&quot;bert-large-nli-stsb-mean-tokens&quot;)
}

This happens with any model that is allocated in GPU. On my NVIDIA GTX 1060 it reaches the 4th cycle, but on smaller GPUs it crashes earlier. One temporal solution is to create the model outside only once, and then pass the model as a parameter to the function as many times as wanted, but I would rather avoid that because it adds an extra step and in any case calling multiple models might just make it crash as well.
Expected behaviour
The for loop finishes without an error
Observed behaviour
Error in py_call_impl(callable, dots$args, dots$keywords) :
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 6.00 GiB total capacity; 2.95 GiB already allocated; 16.11 MiB free; 238.68 MiB cached)
Unsuccesful attemps at solving it

The solutions proposed here
Using numba as suggested here
Declaring the variable explicitely on Python via reticulate::py_run_string() and then doing del bert_encoder and calling the garbage collector

Details
Windows 10 Home
Python 3.7.4
R 4.0.1
Reticulate 1.16
Torch 1.3.1
Tensorflow 2.2.0
Transformers 2.11.0
sentence_transformers 0.2.6

Ok so I am posting my solution for anyone else having this issue.
After each call to the model as
sentence_transformers &lt;- import(&quot;sentence_transformers&quot;)
encoder &lt;- sentence_transformers$SentenceTransformer(&quot;bert-large-nli-stsb-mean-tokens&quot;)

I release GPU memory using
  # Has this been done on a GPU?
  py &lt;- reticulate::py_run_string(&quot;import torch
is_cuda_available = torch.cuda.is_available()&quot;)

  # Release GPU
  if (isTRUE(reticulate::py$is_cuda_available)) {

    tryCatch(reticulate::py_run_string(&quot;del encoder&quot;),
             warning = function(e) {},
             error = function(e) {})

    tryCatch(rm(encoder),
             warning = function(e) {},
             error = function(e) {})

    gc(full = TRUE, verbose = FALSE)

    py &lt;- reticulate::py_run_string(&quot;import torch
torch.cuda.empty_cache()&quot;)

  }

and it works perfectly.
