Customize operation/function on Theano
Question: I am new to the Theano library, which is used for deep learning on GPU device.
I have noticed that there are several build-in operations which can support
gpu computation (I guess they are specially written in a way to support GPU):

    
    
     import theano.tensor as T
     T.sum(), T.neq(), T.argmax(),T.grad()
    

1.What's the difference if I use python's build-in function sum() instead of
T.sum(). Will sum() still work but maybe slower?

  2. Suppose sum() doesn't work for gpu computing, then if I need any operation/function that workable on gpu, I need to implement it in such a way. e.g. I want calculate sin(x) where x is a vector or matrix, and stored in GPU memory. Is there any hint to implement sin(x) that can operate on gpu device? (this might not be suitable or easy to answer)

  3. I have trouble understanding T.grad(). How T.grad can do the symbolic calculation for any given smooth symbolic function? I am very curious about it.


Answer: In theano you have to use `T.sum(), T.neq(), T.argmax(),T.grad()` for symbolic
computation with theano variables like `T.matrix`. You can't use built in
`sum()` for example. If you use theano you have to follow theano's own methods
because theano uses a different form of computing to utilize the gpu's
architecture.

However if you want to use `sum()` you can do the computation with it, and
then create a `theano.shared` variable where you can store the results in,
this way you storing it in the gpu's memory at runtime.

Regarding T.grad() perhaps you should ask the theano developers. :) However I
think when theano is running, it can compute the function's gradient runtime,
using the actual variables utilizing the gpu's computing capacity. I hope this
can help.

**Glad to have been of help! Feel free to accept my answer if you feel it was
useful to you. :-)**

