checking shared libraries for non default loaders
Question: `ldd` is a good simple way to check for shared libraries a given executable is
or will be using. However it does not always work as expected. For example,
see the following shell snippet that demonstrates how it "fails" to found the
libreadline "dependency" into the python binary

I've tried many other distributions, but I'm copying from Tikanga

    
    
    $ lsb_release -a
    LSB Version:    :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch
    Distributor ID: RedHatEnterpriseServer
    Description:    Red Hat Enterprise Linux Server release 5.6 (Tikanga)
    Release:        5.6
    Codename:       Tikanga
    

See what `ldd` does on the default installed `python` (from official
repositories).

    
    
    $ which python
    /usr/bin/python
    $ ldd `which python`
        libpython2.4.so.1.0 => /usr/lib64/libpython2.4.so.1.0 (0x00000030e6200000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00000030e0e00000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00000030e0a00000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00000030ee800000)
        libm.so.6 => /lib64/libm.so.6 (0x00000030e0600000)
        libc.so.6 => /lib64/libc.so.6 (0x00000030e0200000)
        /lib64/ld-linux-x86-64.so.2 (0x00000030dfe00000)
    $ ldd `which python` | grep readline
    $
    

Nothing found about readline. Now I do know from interactive usage that this
binary does have realine functionality, so let't try to see where it's coming
from.

    
    
    $ python &
    [1] 21003
    $ Python 2.4.3 (#1, Dec 10 2010, 17:24:35) 
    [GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    
    [1]+  Stopped                 python
    

Started an interactive python session in background (pid 21003)

    
    
    $ lsof -p 21003
    COMMAND   PID    USER   FD   TYPE DEVICE     SIZE    NODE NAME
    python  21003 ddvento  cwd    DIR   0,33    16384  164304 /glade/home/ddvento/loader-test
    python  21003 ddvento  rtd    DIR    8,3     4096       2 /
    python  21003 ddvento  txt    REG    8,3     8304 6813419 /usr/bin/python
    python  21003 ddvento  mem    REG    8,3   143600 8699326 /lib64/ld-2.5.so
    python  21003 ddvento  mem    REG    8,3  1722304 8699327 /lib64/libc-2.5.so
    python  21003 ddvento  mem    REG    8,3   615136 8699490 /lib64/libm-2.5.so
    python  21003 ddvento  mem    REG    8,3    23360 8699458 /lib64/libdl-2.5.so
    python  21003 ddvento  mem    REG    8,3   145824 8699445 /lib64/libpthread-2.5.so
    python  21003 ddvento  mem    REG    8,3   247544 6821551 /usr/lib64/libreadline.so.5.1
    python  21003 ddvento  mem    REG    8,3    15840 8699446 /lib64/libtermcap.so.2.0.8
    python  21003 ddvento  mem    REG    8,3  1244792 6833317 /usr/lib64/libpython2.4.so.1.0
    python  21003 ddvento  mem    REG    8,3    18152 8699626 /lib64/libutil-2.5.so
    python  21003 ddvento  mem    REG    8,3 56446448 6832889 /usr/lib/locale/locale-archive
    python  21003 ddvento  mem    REG    8,3    21808 6965997 /usr/lib64/python2.4/lib-dynload/readline.so
    python  21003 ddvento  mem    REG    8,3    25464 6901074 /usr/lib64/gconv/gconv-modules.cache
    python  21003 ddvento    0u   CHR  136,1                3 /dev/pts/1
    python  21003 ddvento    1u   CHR  136,1                3 /dev/pts/1
    python  21003 ddvento    2u   CHR  136,1                3 /dev/pts/1
    $ lsof -p 21003 | grep readline
    python  21003 ddvento  mem    REG    8,3   247544 6821551 /usr/lib64/libreadline.so.5.1
    python  21003 ddvento  mem    REG    8,3    21808 6965997 /usr/lib64/python2.4/lib-dynload/readline.so
    

Bingo! Here it is readline!

However, this technique works only when the library is effectively loaded, so
for example it does not find `/usr/lib64/libtcl8.4.so` until the python
process does not run something like `from Tkinter import *`

So I have two questions:

  1. I believe the problem with `ldd` is that it assumes the usage of the standard loader, whereas very likely python is using its own special loader (so that you don't have to relink the executable every time you install a new python module that is not pure python but has some c/c++/fortran code). Is this correct?

  2. Clearly, if an executable is using its own loader, there is no obvious answer to the question "how to find all the possible libraries this executable may load": it depends on what the loader does. But is there a way to find out what libraries may be loaded by python?

PS: related to 1. If you are landing on this question you should already know
the following, but if don't you should: see how simple is to completely mess
up `ldd` output (messing it up only partially is a little harder):

    
    
    $ cat hello.c 
    #include <stdio.h>
    
    int main() {
      printf("Hello world.\n");
      return 0;
    }
    
    $ gcc -static hello.c -o loader
    $ gcc -Wl,--dynamic-linker,./loader hello.c -o hello
    $ ./hello 
    Hello world.
    $ ldd ./hello
    Hello world.
    


Answer: Python, Perl, and other interpreted languages do load things dynamically using
`dlopen()`. (This is not the same thing as replacing the standard loader; they
are still using that, and in fact `dlopen()` is a hook into the standard
loader on ELF-based systems.)

There is no standard registry for loadable modules. Python uses its own rules
to determine where extension modules can be loaded from (look at `sys.path`),
including those which have associated shared objects. Perl uses different
rules. Apache uses still different rules, etc.

So to summarize the answers to your questions:

  1. not exactly

  2. no

