tf.gradient acting like tfp.math.diag_jacobian
I suppose I have found a solution:
my_grad = tf.gradients(tf.reduce_sum(loss), input)
ensures, that the cross dependencies i!=j are ignored - that works really nicely and fast..
I try to calculate noise for input data using the gradient of the loss function from the input-data:
my_grad = tf.gradients(loss, input)
loss is an array of size (n x 1) where n is the number of datasets, m is the size of the dataset, input is an array of (n x m) where m is the size of a single dataset.
I need my_grad to be of size (n x m) - so for each dataset the gradient is calulated. But by definition the gradients where i!=j are zero - but tf.gradients allocates huge amount of memory and runs for prettymuch ever...
A version, which calulates the gradients only where i=j would be great - any Idea how to get there?