title: Meaning of TensorFlow operation ` IsExpensive()`?
body: <p>There is a <a href="https://github.com/tensorflow/tensorflow/blob/754048a0453a04a761e112ae5d99c149eb9910dd/tensorflow/core/framework/op_kernel.h#L103-L106" rel="noreferrer">method in <code>OpKernel</code></a></p>
  
  <pre><code> // Returns true iff this op kernel is considered "expensive". The
   // runtime may use this flag to optimize graph execution for example
   // to "inline" inexpensive kernels.
   virtual bool IsExpensive() { return expensive_; }
  </code></pre>
  
  <p>It seems that by default all operations on the <a href="https://github.com/tensorflow/tensorflow/blob/754048a0453a04a761e112ae5d99c149eb9910dd/tensorflow/core/framework/op_kernel.cc#L99" rel="noreferrer">GPU are considered as inexpensive</a> whilst CPU, SYSL are flagged as expensive.</p>
  
  <p>It is a bit hard to figure out the definition and effect of <code>expensive</code>. The is no <a href="https://www.tensorflow.org/extend/adding_an_op" rel="noreferrer">information in the guide</a>.</p>
  
  <ol>
  <li>Is there any specific guideline when <code>IsExpensive</code> should be <code>false, true</code>? </li>
  <li>What's the effect if an operation is flagged as expensive? So far I can only tell, that active profiling uses this <a href="https://github.com/tensorflow/tensorflow/blob/575db18083d5437fd7a87ccf3414303498911bb1/tensorflow/core/common_runtime/gpu/gpu_device.cc#L557-L560" rel="noreferrer">just as a hint</a> 
  ? The only place querying this property is <a href="https://github.com/tensorflow/tensorflow/blob/98b9a4e45a559217bc89960b889c130af95c1d1a/tensorflow/core/common_runtime/executor.cc#L2216-L2218" rel="noreferrer">in the scheduler</a> but without explaining what being <em>inline</em> means.</li>
  <li>In conjunction with "1." should I care about it in my custom Ops?</li>
  <li>While it makes sense, that any <code>AsyncOp</code> (like <a href="https://github.com/tensorflow/tensorflow/blob/de0904e90669d345e0feaa89d1e99b7618fb6677/tensorflow/core/kernels/remote_fused_graph_execute_op.cc#L123" rel="noreferrer"><code>RemoteFusedGraphExecuteOp</code></a>) is expensive, <a href="https://github.com/tensorflow/tensorflow/blob/3e51dbba08910c14f714d2c68f4d4d2916a2b7eb/tensorflow/contrib/mpi_collectives/mpi_ops.cc#L1114-L1118" rel="noreferrer"><code>MPIAllgatherOp</code></a> seems to be defined as not expensive. Isn't this a contradiction?</li>
  </ol>
  
  <p>I am asking, because the <a href="https://github.com/tensorflow/tensorflow/blob/754048a0453a04a761e112ae5d99c149eb9910dd/tensorflow/core/kernels/identity_op.h#L35" rel="noreferrer"><code>IdentityOp</code></a> is explicitly marked as inexpensive. I wonder, if I should override this method in my custom ops as well, since each CPU version (even <strong>any</strong> custom code) is flagged as expensive.</p>
  
  <p>The entire logic of XLA seems to be about wether <a href="https://github.com/tensorflow/tensorflow/blob/3b538660b1eb22e52ad455a17e01598508373969/tensorflow/compiler/xla/service/instruction_fusion.cc#L50-L153" rel="noreferrer">an instruction is expensive</a> or not. So it might be an important part to consider. Therefore, a coin-toss about true/false might be not the best way to decide the return value in my custom op.</p>