Building Python and more on missing modules
Question: I have another thread asking help on "missing zlib". With the nice help the
problem has been resolved (almost).

Now I am interested in building Python myself (on Ubuntu 10.10).

A few important questions have caught my attention:

  1. After building Python (say 2.7.1), do I need to rebuild Python if I have missing modules?

  2. Is there a way to find out what modules will be missing prior to building Python? Say sqlite3. I have sqlite3 installed for the system default (Python 2.6.6), and I can import that into Python 2.6.6 shell. Now I use pythonbrew to build 2.7.1, and in the shell I cannot import sqlite3 because _sqlite3 is not available. I am sure there are a few more important one missing which I need for future development (such as Django..). 

I am willing to learn how to build without using
[pythonbrew](https://github.com/utahta/pythonbrew).

Please share with me your experience in building another version of Python,
and how would you address the problem of missing modules? Is there a practical
solution to building Python?

I have never bothered building one myself, so please bear with me. I am
beginning to realize the importance of learning and building one myself! Thank
you very much!

* * *

**EDIT**

First I thank you all of your inputs. They meant a lot. I did the building.

    
    
    Python build finished, but the necessary bits to build these modules were not found:
    _bsddb             _curses            _curses_panel   
    _tkinter           bsddb185           bz2             
    dbm                gdbm               readline        
    sunaudiodev        _sqlite3                                    
    To find the necessary bits, look in setup.py in detect_modules() for the module's name.
    

I got sqlite3 and readline away by

    
    
    sudo apt-get install libreadline6 libreadline6-dev
    sudo apt-get install libsqlite3-dev
    

I tried to import them, but still "no named module xxxx".

At [AskUbuntu](http://askubuntu.com/questions/45905/terminator-command-
history) I actually asked people how to get previous commands because I
couldn't use that feature when I am in Python 2.7.1 shell. I believe it's due
to readline.
[Readline](http://ubuntuforums.org/showpost.php?p=6774301&postcount=10)

I installed the Python-2.7.1 under this directory: /home/jwxie518/python27/

I looked into setup.py, I found the following lines:

    
    
    # The sqlite interface
    sqlite_setup_debug = False   # verbose debug prints from this script?
    
    # We hunt for #define SQLITE_VERSION "n.n.n"
    # We need to find >= sqlite version 3.0.8
    sqlite_incdir = sqlite_libdir = None
    sqlite_inc_paths = [ '/usr/include',
                         '/usr/include/sqlite',
                         '/usr/include/sqlite3',
                         '/usr/local/include',
                         '/usr/local/include/sqlite',
                         '/usr/local/include/sqlite3',
                       ]
    

All the paths listed above do not exist. So I guess I have to install sqlite3
manually? I got another reference
[here](http://gcxieblog.blog.163.com/blog/static/56837839200911105418606/)
(it's in Chinese, however)

    
    
    # Download the latest and extract
    # Go into the extracted directory
    ./configure --prefix=/home/jwxie518/python27/python
    make && make install
    # Then edit python-2.7 's setup.py before rebuild it
    # Sample (add these two lines to the end....)
    '~/share/software/python/sqlite-3.6.20/include',
    '~/share/software/python/sqlite-3.6.20/include/sqlite3',
    
    # Then rebuild python like how we did before
    

I went into my directory where I installed sqlite3. I found
**include/sqlite3.h** only. So I went back and check **/usr/include/**. I can
only find sqlite3.h too.

So what is going on here? Readline is also non-importable.

* * *

**3RD EDIT** I started everything over, except I didn't reinstall sqlite3.

    
    
    # Extract Python-2.7.1
    # cd into Python-2.7.1
    # ./configure
    make >make.out 2>&1
    less make.out
    

make.out is here: <http://pastebin.com/raw.php?i=7k3BfxZQ>

I still couldn't import sqlite3. So I went into setup.py and made changes:

    
    
    # We hunt for #define SQLITE_VERSION "n.n.n"
    # We need to find >= sqlite version 3.0.8
    sqlite_incdir = sqlite_libdir = None
    sqlite_inc_paths = [ '/usr/include',
                         '/usr/include/sqlite',
                         '/usr/include/sqlite3',
                         '/usr/local/include',
                         '/usr/local/include/sqlite',
                         '/usr/local/include/sqlite3',
                         '/home/jwxie518/python-mod/include/sqlite',
                         '/home/jwxie518/python-mod/include/sqlite3',
                       ]
    

Then again, ran everything over (this time I also did **make clean**)

Output is here: <http://pastebin.com/raw.php?i=8ZKgAcWn>

According to the output, I don't think the custom path is included.... (for
complete output please go to the link above and search for sqlite)

> build/temp.linux-i686-2.7/home/jwxie518/Python-2.7.1/Modules/_sqlite/util.o
> -L/usr/lib -L/usr/local/lib -Wl,-R/usr/lib -lsqlite3 -o
> build/lib.linux-i686-2.7/_sqlite3.so

I still cannot import sqlite3.

THanks!

* * *

Thank you very much, Michael Dillon, for helping me out. Your tutorial was
neat and clear.

I solved the problem as soon as I realized whenever I tried Python-2.7.1, I
was actually using the one installed by Pythonbrew.

The moral of the story is read all the errors. I neglected the errors
generated by importing sqlite3. The one installed by Pythonbrew didn't have
sqlite3 installed. The development package for sqlite3 was installed after
Pythonbrew installed the Python-2.7.1

Thanks.


Answer: Here is how to build Python and fix any dependencies. I am assuming that you
want this Python to be entirely separate from the Ubuntu release Python, so I
am specifying the --prefix option to install it all in /home/python27 using
the standard Python layout, i.e. site-packages instead of dist-packages.

    
    
    1. Get the .tar.gz file into your own home directory.
    2. tar zxvf Py*.tar.gz
    3. cd Py*1
    4. ./configure --prefix=/home/python27
    5. make
    6. make install
    

Step 5 is the important one. At the end, it will display a list of any modules
that could not be built properly. Often you can fix this by installing an
Ubuntu package, and rerunning make.

    
    
    a. sudo apt-get install something-dev
    b. make
    

It is pretty common to have a problem because you are missing the -dev addon
to some module or other. But sometimes you should start over like this:

    
    
    a. make clean
    b. ./configure --prefix=/home/python27
    c. make
    

Starting over never hurts if you are unsure. An important note about step 6. I
am not using sudo on this command which means that you will need to have the
/home/python27 directory already created with the appropriate ownership.

Don't hesitate to try out `./configure --help |less` before building something
because there may be interesting options that you could use. One time on a
minimal distro I had to do --with-dbmliborder=gdbm:bdb in order to get gdbm
working. When you run ./configure, the last few lines will tell you where it
put the information that it learned. In the case of Python, Modules/Setup has
been useful to figure out how to get a module to build.

Another useful thing is to `make clean` and then run `make >make.out 2>&1` to
capture all the output from the full make process. Then, after it is complete,
use `less` or an editor to look for the details on a problem module such as
_sqlite. For instance, check all the -I options that are passed to `gcc`. If
the correct include directory is not on the list that would cause a problem.
You can edit `setup.py` to change the list of include directories.

In the past it was more common to have library problems that would be fixed by
logging out, logging in again, and running "sudo ldconfig" before doing a
complete rebuild.

