Deep learning : CPU vs. GPU quality I searched for other similar questions on Google and here, but couldn't find any.
I'm consistently getting results of less quality when making deep learning with GPU, compared to CPU. Of course, it is much faster using GPU, but my results converge to lower quality on many different aspects: subjective quality of embedding, validation accuracy, loss, etc. It doesn't depend on the number of epochs, ie. results won't converge toward the same value: one is always inferior to the other. It's disconcerting when trying to achieve solid, reproducible SOTA results, where .1 or .2% is quite important (litteraly from top2 to top1).

I have a RTX2070 and a good CPU. Is it inherent to modern GPU ? Are approximations done using GPU that could explain this phenomenon ? Thanks for your inputs.
In theory, CPU and GPU should reach same accuracy.
But imo, generally people tend to use bigger batch size on GPU due to high parallelization. As you might know, in stochastic gradient descent a smaller batch size has a higher chance of reaching the global optimum. This could be a reason for the slight difference in accuracy.