TensorFlow while_loop parallelization TensorArray
You should probably get the parallel execution of the first 5 iterations and the second 5 iterations. I can say for sure if you provide a code sample.
I don't exactly understand how the while_loop parallelization works. Suppose I have a TensorArray having 10 Tensors all of same shape. Now suppose the computations in the loop body for the first 5 Tensors are independent of the computations in the remaining 5 Tensors. Would TensorFlow run these two in parallel? Also if I use a Tensor instead of a TensorArray and made the updates to it using scatter_update, would it pass the gradients properly during backprop?