How to limit GPU memory usage when only prediction in c++?
@powermew,
Could you please take a look at this official code link which helps to limit the gpu on C++.
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/config.proto

The _allocated part of set has to do with protobuf's memory management. If you set the options, it will expect an object that was dynamically allocated and it will take ownership over it (and delete it when appropriate). Your code won't work, because you're passing an address to a local variable that will get destroyed when you exit the code block.

Instead, if you access the existing instance of GpuOptions via mutable_gpu_options() you get a pointer to the object that you can use to edit directly that instance.

```
auto options = tensorflow::SessionOptions();
options.config.mutable_gpu_options()->set_per_process_gpu_memory_fraction(0.2);
options.config.mutable_gpu_options()->set_allow_growth(true);
tensorflow::Status status = tensorflow::NewSession(options, &session);
```
https://github.com/serizba/cppflow/issues/102

Thank you! > @powermew, Could you please take a look at this official code link which helps to limit the gpu on C++. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/config.proto
> 
> The _allocated part of set has to do with protobuf's memory management. If you set the options, it will expect an object that was dynamically allocated and it will take ownership over it (and delete it when appropriate). Your code won't work, because you're passing an address to a local variable that will get destroyed when you exit the code block.
> 
> Instead, if you access the existing instance of GpuOptions via mutable_gpu_options() you get a pointer to the object that you can use to edit directly that instance.
> 
> ```
> auto options = tensorflow::SessionOptions();
> options.config.mutable_gpu_options()->set_per_process_gpu_memory_fraction(0.2);
> options.config.mutable_gpu_options()->set_allow_growth(true);
> tensorflow::Status status = tensorflow::NewSession(options, &session);
> ```
> 
> [serizba/cppflow#102](https://github.com/serizba/cppflow/issues/102)
> 
> Thank you!

Thank you for reply
I solved this problem with adding code from below comment.
https://github.com/Neargye/hello_tf_c_api/issues/21#issuecomment-505055121 @powermew,
Glad the issue was resolved. If there are no more open items for this issue, could you please feel free to move this issue to closed status. Thank you!
### Issue type

Feature Request

### Have you reproduced the bug with TensorFlow Nightly?

No

### Source

source

### TensorFlow version

tf 2.4

### Custom code

Yes

### OS platform and distribution

Windows 10

### Mobile device

_No response_

### Python version

3.7.9

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

11.0 / 8.1.0

### GPU model and memory

RTX 3090 / 24GB

### Current behavior?

I loaded the saved model using the already compiled tensorflow-gpu 2.4.0.
When this model was used for prediction, it was confirmed that all available memory of the gpu was used.
I've seen limiting using the growing method in python, but I don't know how to use it in c++. could you please tell me how?



### Standalone code to reproduce the issue

```shell
#include <stdlib.h>
#include <stdio.h>
#include <tensorflow/c/c_api.h>
#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <vector>
void NoOpDeallocator(void* data, size_t a, void* b) {}

int main() {
    TF_Graph* Graph = TF_NewGraph();
    TF_Status* Status = TF_NewStatus();

    TF_SessionOptions* SessionOpts = TF_NewSessionOptions();
    TF_Buffer* RunOpts = NULL;

    const char* saved_model_dir = "H:\\my_model\\"; // Path of the model
    const char* tags = "serve"; // default model serving tag; can change in future
    int ntags = 1;

    TF_Session* Session = TF_LoadSessionFromSavedModel(SessionOpts, RunOpts, saved_model_dir, &tags, ntags, Graph, NULL, Status);
    if (TF_GetCode(Status) == TF_OK)
    {
        printf("TF_LoadSessionFromSavedModel OK\n");
    }
    else
    {
        printf("%s", TF_Message(Status));
    }
}
```


### Relevant log output

_No response_