Is it a good idea to use tensorflow instead of numpy for numerical approximations? I intend on perfroming some numerical approximations for a problem in physics. 

The main gist of the program will be to perform a svd on large sparse matrices and also calculating the trace of a large matrix. 

I used numpy/scipy and multiprocessing modules in python to do this, but it is not fast enough.

I also implemented the same code using tensorflow on some gpus. Tensorflow does the calulations much faster, but it takes a long time to send the data into the gpus and bring it out. Each instance of data is sent individually as float values.  Is there any way to send all the data at once into the gpu?

Is it a good or bad idea to use tensorflow for numerical calculations? 

Below is a snippet of my code....

def Expectation_Value(density,N):
   expec=0
   for i in np.arange(1,N+1):
       a=tf.linalg.trace(tf.matmul(Sigma(i,N,Sz),density))
       expec+=a
return expec/N

def main():
   df=pd.read_csv('.....')
   sess=tf.Session()

   for l in range(len(df)):
   Delta=df['Delta'].iloc[l]
   Omega=df['Omega'].iloc[l]
   Gamma=df['Gamma'].iloc[l]
   J=df['J'].iloc[l]


    Sz_tens=tf.zeros(len(df))
    Expec=Expectation_Value(DMT,N)
    Sz_tens[l]=Expec
Sz_arr=Sz_tens.eval(session=sess)
df['Sz']=Sz_arr
sess.close()
df.head()
TensorFlow is a library that supports dataflow paradigm which makes it a good candidate for numerical calculations.

In general, TensorFlow data loading performance can be optimized using tf.data module. In particular, there is a concept called GPUs starvation, where GPUs are waiting for data to be loaded. GPU starvation can be minimized in TensorFlow with Pipelining, overlapping the preprocessing and model execution of a training step.

In particular, your code can be optimized by refactoring to Expectation_Value operate on vectors instead of scalars. Right now N and i are scalars and there is a for-loop that processes elements one at a time. tf.linalg.trace can take a tensor as an input so that N and i could be vectors. The function will be much faster once it operates on vectors and no longer uses a for-loop.