CUDA_ERROR_OUT_OF_MEMORY
Did you try to start your training with a smaller data set and that worked?

How about using generators to gradually input your data? For me that worked quite well (in Keras : model.fit_generator(...))