What is the right way to manage namespaces in Python 3?
Question: Most of my programming background is in C++ and Java, but for professional
reasons I'm starting to learn Python. One of the first things that I noticed
was Python's new approach to packages and namespaces, but after googling
around for awhile and doing some experimenting I thought I started to get the
hang of it.

Then I read [this](http://python-
notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html),
which indicated that everything I had just learned was wrong. So now I'm
confused again, and I'm having trouble figuring out how I should organize my
code.

* * *

Here's a very concrete question which I don't really know how to answer
correctly for any version of Python, and in particular I don't know if the
answer changed from Python 2 to 3. Let's say I want a class in a cloned github
repo, stored in:

    
    
    /Users/me/my_teams_code/some_project/package/IWantThisClass.py
    

But let's say I have some other projects here:

    
    
    /Users/me/my_own_code/some_other_project1/...
    /Users/me/my_own_code/some_other_project2/...
    /Users/me/my_own_code/some_other_project3/...
    

each of which use IWantThisClass. One strategy seems to be to just manually
add the correct directory to sys.path every time, but this is rather
cumbersome - especially when I'm just experimenting in a Jupyter notebook. The
solution that I settled upon after googling around is to add the package
directory to my PYTHONPATH variable in bash; this works in the sense that
"import IWantThisClass" works just fine. But I read in Coghlan's post above:

> "This next trap exists in all current versions of Python, including 3.3, and
> can be summed up in the following general guideline: 'Never add a package
> directory, or any directory inside a package, directly to the Python path'."

So what is the "correct" way to solve this problem?


Answer: Basically, if `package` is the top-level directory of your package, you should
add the directory _containing_ `package` to `sys.path`. If what you want to
import is not a package but a single module (i.e., just an individual `.py`
file not in a package directory), then that module should be in a directory on
`sys.path`/

Exactly how you should add these paths is another question. For situations
where I want to add one-off things to the path, I find that the easiest thing
to do is to create a `.PTH` file inside a "site directory" --- either the main
site-packages directory or the user-specific site-packages directory. In that
`.PTH` file you can list other directories which will then be added to
`site.path`. So you can have your own little `my_stuff.pth` file that sets up
`sys.path` with directories you need. This avoids having to add boilerplate
code to each script to manually modify `sys.path`. You can read about how to
do this in the documentation for the [`site`
module](https://docs.python.org/2/library/site.html).

Another, perhaps better answer is that you should try to avoid adding specific
directories to `sys.path` in order to access specific files. Rather, you
should try to _install_ the relevant packages, which will let Python set up
the paths for you. If you're using in-development packages that you don't want
to clutter your main site-packages will, you can use [`python setup.py
develop`](https://pythonhosted.org/setuptools/setuptools.html#development-
mode) or [`pip install
--editable`](https://pip.pypa.io/en/latest/reference/pip_install.html#editable-
installs) to install the packages "in-place". (That is, the package files will
stay where they are, not be moved into the global Python site-packages, and
the installer will update the appropriate `.PTH` files so that the newly-
installed package is importable.)

In short, ultimately you need to move away from viewing it in terms of "this
file wants this class from this other file". The unit that you should be
concerned with is the installable package or module. If you're writing code
that wants SomeClass, you should install the package/module that provides
SomeClass and then import it; don't try to manipulate `sys.path` on an ad-hoc
basis to target the individual file/class that you want without installing its
package. (As mentioned, `setup.py develop` and `pip install --editable` allow
you to do this while still keeping the actual files where you want.)

