title: Change number of threads for Tensorflow inference with C API
body: <p>I'm writing a c++ wrapper around tensorflow 1.2 C API (for inference purposes, if it matters). 
  Since my application is a multi-process and multi-threaded one, where resources are explicitly allocated, I would like to limit Tensorflow to only use one thread.</p>
  
  <p>Currently, running a simple inference test that allows batch processing, I see it is using all CPU cores.
  I have tried limiting number of threads for a new session using a mixture of C and C++ as follows (forgive my partial code snippet, I hope this makes sense):</p>
  
  <pre><code>tensorflow::ConfigProto conf;
  conf.set_intra_op_parallelism_threads(1);
  conf.set_inter_op_parallelism_threads(1);
  conf.add_session_inter_op_thread_pool()-&gt;set_num_threads(1);
  std::string str;
  conf.SerializeToString(&amp;str);
  TF_SetConfig(m_session_opts,(void *)str.c_str(),str.size(),m_status);
  m_session = TF_NewSession(m_graph, m_session_opts, m_status);
  </code></pre>
  
  <p>But I don't see it is making any difference - all cores are still fully utilized.</p>
  
  <p>Am I using the C API correctly?</p>
  
  <p>(My current work around is to recompile Tensorflow with hard coding number of threads to be 1, which will probably work, but its obviously not the best approach...)</p>
  
  <p>-- Update --</p>
  
  <p>I also tried adding:</p>
  
  <pre><code>conf.set_use_per_session_threads(true);
  </code></pre>
  
  <p>Without success. Still multiple cores are used...</p>
  
  <p>I also tried to run with high log verbosity, and got this output (showing only what I think is relevant):</p>
  
  <pre><code>tensorflow/core/common_runtime/local_device.cc:40] Local device intraop parallelism threads: 8
  tensorflow/core/common_runtime/session_factory.cc:75] SessionFactory type DIRECT_SESSION accepts target: 
  tensorflow/core/common_runtime/direct_session.cc:95] Direct session inter op parallelism threads for pool 0: 1
  </code></pre>
  
  <p>The "parallelism threads: 8" message shows up as soon as I instantiate a new graph using TF_NewGraph(). I didn't find a way to specify options prior to this graph allocation though...</p>