How to run a model in an application using gpu (without CUDA) I am looking for a way to utilize a computer’s gpu without using cuda (or any installable software). The reason for this is I have an application where the neural network runs on a user’s computer and I can’t assume they will have the knowledge/ spend the time and effort to install CUDA. I see that it is possible to utilize the gpu using WebGL but from my reading, it sounds like due to other limitations with this approach it is not actually faster than using cpu. Does anyone know how to do this maybe using openCL or some other way to do it?
I'm pretty sure that you will need CUDA to use the GPU, given you have included the tag tensorflow. All of the ops in tensorflow are written in C++, which the uses the CUDA API to speak to the GPU. Perhaps there are libraries out there for performing matrix multiplication on the GPU without CUDA, but I haven't heard of a deep learning framework that doesn't use CUDA, when executing on the GPU.

What you can perhaps do it find a solution that works using Docker. The latest version doesn't require any additional Nvidia software to be installed. It would mean your customer doesn't need to mess around with CUDA themselves, they just need docker installed, which is generally less than 5 minutes work. Here is the Tensorflow documentation for a starter.

In addition, using Docker is perhaps the most professional and isolated way to deliver code to somebody else's machine. There are many benefits and it works on Windows, Mac, Linux. Any changes you make can simply be pushed to their machine, without them having to change anything. It can also scale arbitrarily to multiple machines, running locally or in the cloud, the list goes on and on...