title: Problems with a local installation of libc
body: <p>I'm trying to run a Theano implementation of alexNet on some machines at work. When I first tried to run it I got the following error:</p>
  
  <pre><code>$ THEANO_FLAGS=mode=FAST_RUN,floatX=float32 python train.py
  
   ....
  
   ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /data1/software/analysis/libstdc++.6.0.15/usr/lib64/libstdc++.so.6)
  </code></pre>
  
  <p>Normally, what I would do in this situation is use my package manager to update libc, but this is not an option for work/administrative reasons. So, what I did instead was to install a local version of libc and to change LD_LIBRARY_PATH in my .bashrc file to point to it. Now, however, I get the following error:</p>
  
  <pre><code>$ THEANO_FLAGS=mode=FAST_RUN,floatX=float32 python train.py
  python: relocation error: /data1/software/analysis/libc.2.14/lib64/libc.so.6: symbol _dl_starting_up, version GLIBC_PRIVATE not defined in file ld-linux-x86-64.so.2 with link time reference
  </code></pre>
  
  <p>Oddly enough, I get a similar 'relocation error' when I just try to 'ls'.</p>
  
  <p>Does anyone know how I can install a version of libc that will only be used by my Python interpreter, but not by everything else?</p>
  
  <p>Note: The libstdc++ I'm using is a locally installed version. I installed it for the same reason that I'm trying to install a local version of libc. </p>
  
  <p>...</p>
  
  <p>OK, I've progressed a little further, but am still stuck. </p>
  
  <p>If I return to my old LD_LIBRARY_PATH, in order to avoid errors with commands like 'ls', I can verify that the new (local) libc does, in fact, use the old (system) ld-linux-x86-64.so.2</p>
  
  <pre><code>$ ldd /data1/software/analysis/libc.2.14/lib64/libc.so.6
  /lib64/ld-linux-x86-64.so.2 (0x0000003b9d400000)
  linux-vdso.so.1 =&gt;  (0x00007fff887ff000)
  </code></pre>
  
  <p>So, I thought I would change LD_LIBRARY_PATH to look for the new (local) ld-linux-x86-64.so.2</p>
  
  <pre><code>$ export LD_LIBRARY_PATH=/data1/software/analysis/libc.2.14:$LD_LIBRARY_PATH
  $ ls /data1/software/analysis/libc.2.14/lib64
    ld-2.14.1.so  libanl-2.14.1.so   libcidn.so.1
    libm-2.14.1.so       libnss_compat.so.2    libnss_hesiod-2.14.1.so 
    libnss_nisplus.so.2   librt-2.14.1.so libutil.so.1
    ld-linux-x86-64.so.2     libanl.so.1        libcrypt-2.14.1.so   
    libm.so.6        libnss_dns-2.14.1.so    libnss_hesiod.so.2
    libpthread-2.14.1.so  librt.so.1      rtkaio
    libBrokenLocale-2.14.1.so  libc-2.14.1.so     libcrypt.so.1     
    libnsl-2.14.1.so     libnss_dns.so.2     libnss_nis-2.14.1.so  
    libpthread.so.0      libthread_db-1.0.so  
    libBrokenLocale.so.1     libc.so.6          libdl-2.14.1.so        
    libnsl.so.1          libnss_files-2.14.1.so  libnss_nis.so.2         
    libresolv-2.14.1.so   libthread_db.so.1
    libSegFault.so           libcidn-2.14.1.so  libdl.so.2      
    libnss_compat-2.14.1.so  libnss_files.so.2       libnss_nisplus-2.14.1.so 
    libresolv.so.2       libutil-2.14.1.so
  
  $ ldd /data1/software/analysis/libc.2.14/lib64/libc.so.6
  /lib64/ld-linux-x86-64.so.2 (0x0000003b9d400000)
  linux-vdso.so.1 =&gt;  (0x00007fffad5ff000)
  
  $ echo $LD_LIBRARY_PATH
    /data1/software/analysis/libc.2.14:
    /data1/software/analysis    
    /libstdc++.6.0.15/usr/lib64:/data1/software/analysis/anaconda/lib:
    /data2/software/stable/CUDA/5.5.22/lib:
    /data2/software/stable/CUDA/5.5.22/include:
    /data2/software/stable/CUDA/5.5.22/lib64:
    /data1/software/analysis/anaconda/lib:
    /data2/software/stable/CUDA/5.5.22/lib:
    /data2/software/stable/CUDA/5.5.22/include:
    /data2/software/stable/CUDA/5.5.22/lib64:
  </code></pre>
  
  <p>So, my libc should see the new ld-linux-x86-64.so.2. I can verify that this so has the symbol _dl_starting_up:</p>
  
  <pre><code>$ nm /data1/software/analysis/libc.2.14/lib64/ld-linux-x86-64.so.2 | grep _dl_start*
    0000000000004c00 t _dl_start
    000000000000fc30 t _dl_start_profile 
    0000000000001698 t _dl_start_user
    0000000000220000 V _dl_starting_up
    0000000000220000 b _dl_starting_up_internal
  </code></pre>
  
  <p>But, when I try to run the alexNet implementation using LD_PRELOAD to guarantee the use of the right libc, I still get the same error:</p>
  
  <pre><code>$ LD_PRELOAD=/data1/software/analysis/libc.2.14/lib64/libc.so.6 THEANO_FLAGS=mode=FAST_RUN,floatX=float32 python train.py
  
  python: relocation error: /data1/software/analysis/libc.2.14/lib64/libc.so.6: symbol _dl_starting_up, version GLIBC_PRIVATE not defined in file ld-linux-x86-64.so.2 with link time reference
  </code></pre>
  
  <p>Why is the new (local) ld-linux not overriding the old (system) ld-linux? Shouldn't my setting of LD_LIBRARY_PATH have taken care of that?</p>
  
  <p>If I try to force the issue, by preloading ld-linux, I get a segmentation fault:</p>
  
  <pre><code>$ LD_PRELOAD=/data1/software/analysis/libc.2.14/lib64/ld-linux-x86-64.so.2:/data1/software/analysis/libc.2.14/lib64/libc.so.6 THEANO_FLAGS=mode=FAST_RUN,floatX=float32 python train.py
  Segmentation fault (core dumped)
  </code></pre>
  
  <p>So, now I'm stuck. I don't know</p>
  
  <ol>
  <li><p>Why my setting of LD_LIBRARY_PATH didn't force my new (local) libc to use the new (local) ld-linux.</p></li>
  <li><p>What I should do about the segmentation fault when I force libc to use the new (local) ld-linux.</p></li>
  </ol>