title: tensorflow: invalid fastbin entry (free): 0x00007f2fa8023940
body: <p>I am training inception model with TensorFlow from scratch following <a href="https://github.com/tensorflow/models/tree/master/inception" rel="nofollow noreferrer">this</a>, this is my environment configuration：</p>
  
  <ul>
  <li>Tensorflow version: 0.11.0rc1 (compile from source)</li>
  <li>OS: CentOS Linux release 7.0.1406 (Core) 64bit</li>
  <li>model: models/inception</li>
  </ul>
  
  <p>But after about 11500 steps got this error:</p>
  
  <pre><code>...
  ...
  2016-11-03 22:37:06.142819: step 11540, loss = 9.38 (66.9 examples/sec; 0.957 sec/batch)
  2016-11-03 22:37:15.753609: step 11550, loss = 9.22 (67.4 examples/sec; 0.950 sec/batch)
  2016-11-03 22:37:25.332004: step 11560, loss = 9.51 (65.6 examples/sec; 0.975 sec/batch)
  *** Error in `/home/software/anaconda2/bin/python': invalid fastbin entry (free): 0x00007f2fa8023940 ***
  ======= Backtrace: =========
  /lib64/libc.so.6(+0x7d19d)[0x7f315d7b919d]
  /home/software/anaconda2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(+0x248ff48)[0x7f314baa2f48]
  /home/software/anaconda2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(+0x244520f)[0x7f314ba5820f]
  /home/software/anaconda2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(_ZN10tensorflow19LocalRendezvousImpl4SendERKNS_10Rendezvous9ParsedKeyERKNS1_4ArgsERKNS_6TensorEb+0xf9)[0x7f314bb9e7f9]
  /home/software/anaconda2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(_ZN10tensorflow22IntraProcessRendezvous4SendERKNS_10Rendezvous9ParsedKeyERKNS1_4ArgsERKNS_6TensorEb+0xb4)[0x7f314ba57b74]
  /home/software/anaconda2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(_ZN10tensorflow6SendOp7ComputeEPNS_15OpKernelContextE+0x346)[0x7f314baa3736]
  /home/software/anaconda2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(+0x242ea59)[0x7f314ba41a59]
  /home/software/anaconda2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(+0x2422e30)[0x7f314ba35e30]
  /home/software/anaconda2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(_ZN5Eigen26NonBlockingThreadPoolTemplIN10tensorflow6thread16EigenEnvironmentEE10WorkerLoopEi+0x3c8)[0x7f314bc474a8]
  /home/software/anaconda2/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so(_ZNSt17_Function_handlerIFvvEZN10tensorflow6thread16EigenEnvironment12CreateThreadESt8functionIS0_EEUlvE_E9_M_invokeERKSt9_Any_data+0x22)[0x7f314bc46c72]
  /home/software/anaconda2/bin/../lib/libstdc++.so.6(+0xb4870)[0x7f3149153870]
  /lib64/libpthread.so.0(+0x7df3)[0x7f315e20ddf3]
  /lib64/libc.so.6(clone+0x6d)[0x7f315d8321ad]
  ======= Memory map: ========
  00400000-00401000 r-xp 00000000 fd:02 34476856                           /home/software/anaconda2/bin/python2.7
  00600000-00601000 rw-p 00000000 fd:02 34476856                           /home/software/anaconda2/bin/python2.7
  0067e000-42ae4000 rw-p 00000000 00:00 0                                  [heap]
  200000000-200100000 rw-s 1026d71000 00:05 221089                         /dev/nvidiactl
  200100000-204100000 ---p 00000000 00:00 0 
  204100000-204200000 rw-s f70ee2000 00:05 221089                          /dev/nvidiactl
  204200000-204300000 ---p 00000000 00:00 0 
  204300000-204400000 rw-s f75483000 00:05 221089                          /dev/nvidiactl
  204400000-204500000 ---p 00000000 00:00 0 
  204500000-204600000 rw-s 1014d38000 00:05 221089                         /dev/nvidiactl
  204600000-208600000 ---p 00000000 00:00 0 
  208600000-208700000 rw-s f7735a000 00:05 221089                          /dev/nvidiactl
  208700000-208800000 ---p 00000000 00:00 0 
  208800000-208900000 rw-s f7777d000 00:05 221089                          /dev/nvidiactl
  208900000-208a00000 ---p 00000000 00:00 0 
  208a00000-208b00000 rw-s f77eaa000 00:05 221089                          /dev/nvidiactl
  208b00000-20cb00000 ---p 00000000 00:00 0 
  ...
  ...
  </code></pre>