Using CPU after training in GPU I am using tensorflow-gpu 1.10.0 and keras-gpu 2.2.4 with a Nvidia gtx765M(2GB) GPU, OS is Win8.1-64 bit- 16GB RAM.

I can train a network with 560x560 pix images and batch-size=1, but after training is over when I try to test/predict I get the following error:

ResourceExhaustedError: OOM when allocating tensor with shape[20,16,560,560] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
     [[Node: conv2d_2/convolution = Conv2D[T=DT_FLOAT, data_format="NCHW", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](activation_1/Relu, conv2d_2/kernel/read)]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.


I suppose it's a memory issue.

So my question is, is it possible to first use GPU for training and then switching CPU to predict some results in one Jupyter notebook?

Can we free up GPU-memory in windows inside a script?

I found these two topics but I think we should use at the beginning of the script. What I want is switching after training.

https://github.com/keras-team/keras/issues/4613

Switching Keras backend Tensorflow to GPU

Any help will be appreciated,
Thanks
For prediction step, are you using the same batch size as training ?  batch prediction should bring down memory usage.