title: `dlopen`'ing a .so that contains symbols results in undefined symbols
body: <h2>Problem</h2>
  
  <p>To be concrete, I am using <code>cblas</code> for my application. I create a <code>foo.so</code> that uses <code>cblas</code> as a subroutine, and I <code>dlopen('foo.so', RTLD_LAZY)</code>.</p>
  
  <p>This is the flags I use to compile(redacted some <code>-I</code> flags):</p>
  
  <pre><code>g++-8 -std=c++17 -shared -O3 -xc++ 
      -ffast-math -fPIC -fopenmp -lcblas - -o /mnt/xxx/python/test.so
  </code></pre>
  
  <p>This is the error I get upon <code>dlopen()</code>:</p>
  
  <pre><code>test.so: undefined symbol: cblas_dgemm
  </code></pre>
  
  <p>Here we see that <code>cblas_dgemm</code> is included in the <code>.so</code>.</p>
  
  <pre><code>root@cd872c4b85ff:/mnt/xxx/python# objdump -TC test.so | grep cblas
  0000000000000000      D  *UND*  0000000000000000              cblas_ddot
  0000000000000000      D  *UND*  0000000000000000              cblas_dsyrk
  0000000000000000      D  *UND*  0000000000000000              cblas_dgemm
  0000000000000000      D  *UND*  0000000000000000              cblas_dgemv
  </code></pre>
  
  <p>But in <code>ldd</code> it is not listed as a dependency.</p>
  
  <pre><code>root@cd872c4b85ff:/mnt/xxx/python# ldd test.so
      linux-vdso.so.1 (0x00007ffdbb3c4000)
      libstdc++.so.6 =&gt; /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f43f23a7000)
      libgomp.so.1 =&gt; /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f43f2177000)
      libgcc_s.so.1 =&gt; /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f43f1f5f000)
      libc.so.6 =&gt; /lib/x86_64-linux-gnu/libc.so.6 (0x00007f43f1b6e000)
      libm.so.6 =&gt; /lib/x86_64-linux-gnu/libm.so.6 (0x00007f43f17d0000)
      /lib64/ld-linux-x86-64.so.2 (0x0000562eb733a000)
      libdl.so.2 =&gt; /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f43f15cc000)
      libpthread.so.0 =&gt; /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f43f13ab000)
  </code></pre>
  
  <p>I did a test compile and it says that <code>-lcblas</code> was successfully found:</p>
  
  <pre><code>g++-8 -lcblas -Wl,--verbose | grep blas
  attempt to open /usr/lib/gcc/x86_64-linux-gnu/8/libcblas.so failed
  attempt to open /usr/lib/gcc/x86_64-linux-gnu/8/libcblas.a failed
  attempt to open /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libcblas.so succeeded
  -lcblas (/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libcblas.so)
  /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/Scrt1.o: In function `_start':
  (.text+0x20): undefined reference to `main'
  collect2: error: ld returned 1 exit status
  </code></pre>
  
  <h2>What does work</h2>
  
  <p>One important thing is that if I set my environment variable for <code>LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libcblas.so</code>, the executable will run correctly. However, this is obviously not The Right Way to get the executable to work. <code>-Wl,rpath=</code> is also not The Right Way, since cblas will reside on different directories (i.e. on OS X it's in <code>/usr/lib/libcblas.so</code>, and in ubuntu it's <code>/usr/lib/x86_64-linux-gnu/libcblas.so</code>, and I'm guessing there will be more unpredictable locations).</p>
  
  <p>What could be the reason? Am I not linking it correctly? Is the dynamic linker not finding the symbols for some reason?</p>
  
  <p>Since some of the information is redacted, please let me know if there is anything that is needed and I'll add it to the information above. Thanks.</p>
  
  <p>EDIT: The symbols do exist inside of <code>libcblas.so</code>:</p>
  
  <pre><code>root@3841c2760800:/usr/lib/x86_64-linux-gnu# nm -D libcblas.so | grep cblas_ddot
  000000000000a0a0 T cblas_ddot
  root@3841c2760800:/usr/lib/x86_64-linux-gnu# nm -D libcblas.so | grep cblas_dgemm
  0000000000005790 T cblas_dgemm
  root@3841c2760800:/usr/lib/x86_64-linux-gnu# nm -D libcblas.so | grep cblas_dgemv
  00000000000077f0 T cblas_dgemv
  root@3841c2760800:/usr/lib/x86_64-linux-gnu# nm -D libcblas.so | grep cblas_dsyrk
  00000000000065f0 T cblas_dsyrk
  </code></pre>
  
  <p>EDIT:</p>
  
  <p>The dependency graph, I believe, looks something like:</p>
  
  <pre><code>main --(dlopens)--&gt; test.so 
      --(includes)--&gt; header-only linear algebra library 
      --(calls cblas subroutine)--&gt; /usr/lib/x86.../libcblas.so
  </code></pre>