title: tensorflow py_func is handy but makes my training step very slow.
body: <p>I have some efficiency issue using the tensorflow function py_func.</p>
  
  <p><strong>Context</strong> </p>
  
  <p>In my project, I have a batch of tensor <code>input_features</code> of size <code>[? max_items m]</code>. The first dimension is set to <code>?</code> because it is a dynamic shape (the batch is read for a custom tensorflow reader, and shuffled using tf.train.shuffle_batch_join()). The second dimension correspond to an upper bound (the maximum number of items I can take for my example), the third dimension corresponds to the feature dimension space. I also have a tensor <code>num_items</code> that has dimension of batch size (so the shape is <code>(?,)</code>), indicating the number of items in the example, other are set to 0 (in a numpy writing style <code>input_feature[k, num_items[k]:, :] = 0</code>)</p>
  
  <p><strong>Issue</strong></p>
  
  <p>My workflow need some custom python operations (especially for dealing with indexing, I need or instance to perform clustering operations on some chunk of examples) and I use a few numpy function wrapped in <code>py_func</code> function. This works well, but training becomes very very slow (around 50 times slower than a model without this py_func), and the function itself is not time consuming. </p>
  
  <p><strong>Questions</strong></p>
  
  <p>1 - Is this computing time increase normal? The function wrapped in <code>py_func</code> gives me a new tensor that is multiplied further in the process. Does it explain the computing time? (I mean gradient may be more difficult to compute with such function).</p>
  
  <p>2 - I'm trying to modify my processing and avoid using <code>py_func</code> function. However, it was very handy for extracting data with numpy indexing (especially with my data formatting), and I have some difficulties to pass it in a TF way. For instance, if I have a tensor <code>t1</code> with shape<code>[-1, n_max, m]</code> (first dimension is batch_size which is dynamic) and <code>t2</code> with shape <code>[-1,2]</code> containing integers. Is there an easy way to perform mean operation in tensorflow that will results in <code>t_mean_chunk</code> with shape <code>(-1, m)</code> where (in a numpy formulation) :
  <code>t_mean_chunk[i,:] = np.mean(t1[i, t2[i,0]:t2[i,1], :], axis=0)</code> ?
  This was (among others operations) the kind of things I was doing in wrapped function.</p>