Does Numpy automatically detect and use GPU?
Does Numpy/Python automatically detect the presence of GPU and utilize
it to speed up matrix computation (e.g. numpy.multiply,
numpy.linalg.inv, ... etc)?

No.

Or do I have code in a specific way to exploit the GPU for fast
computation?

Yes. Search for Numba, CuPy, Theano, PyTorch or PyCUDA for different paradigms for accelerating Python with GPUs.
I have a few basic questions about using Numpy with GPU (nvidia GTX 1080 Ti).  I'm new to GPU, and would like to make sure I'm properly using the GPU to accelerate Numpy/Python.  I searched on the internet for a while, but didn't find a simple tutorial that addressed my questions.  I'd appreciate it if someone can give me some pointers:
1) Does Numpy/Python automatically detect the presence of GPU and utilize it to speed up matrix computation (e.g. numpy.multiply, numpy.linalg.inv, ... etc)?  Or do I have code in a specific way to exploit the GPU for fast computation?
2) Can someone recommend a good tutorial/introductory material on using Numpy/Python with GPU (nvidia's)?
Thanks a lot!