A simple Hello World setuptools package and installing it with pip
Question: I'm having trouble figuring out how to install my package using setuptools,
and I've tried reading the documentation on it and SO posts, but I can't get
it to work properly. I'm trying to get a simple helloworld application to
work. This is how far I got:

helloworld.py:

    
    
    print("Hello, World!")
    

README.txt:

    
    
    Hello, World! readme
    

MANIFEST.in:

    
    
    recursive-include images *.gif
    

setup.py:

    
    
    from setuptools import setup, find_packages
    
    setup(
        name='helloworld',
        version='0.1',
        license='BSD',
        author='gyeh',
        author_email='hello@world.com',
        url='http://www.hello.com',
        long_description="README.txt",
        packages=find_packages(),
        scripts = ['helloworld.py'],
        package_data={
            "" : ["images/*.gif"]
        },
        data_files=[('images', ['images/hello.gif'])],
        description="Hello World testing setuptools",
    )
    

And I have a blank file called images/hello.gif that I want to include in my
package as additional data. The folder structure looks like this:

    
    
    testsetup/  
    |-- helloworld.py  
    |-- images/  
    |-- --- hello.gif  
    |-- MANIFEST.in  
    |-- README.txt  
    |-- setup.py  
    

When I run `python setup.py sdist`, it generates the `dist` and
`helloworld.egg-info` successfully. When I look at SOURCES.txt under egg-info,
it contains the script and the image under the images folder, and the tarball
under dist contains them as well.

However, when I try to run `pip install --user helloworld-0.1.tar.gz` on the
tarball, it successfully installs it, but I can't find the program files
helloworld.py and images/hello.gif.

When I look under `$HOME/.local/lib/python3.3/site-packages/`, I see the egg-
info folder and all of it's contents installed there. But the
`$HOME/.local/bin` folder doesn't even exist. Are the program files stores
elsewhere? What am I doing wrong here? I'm running Arch Linux.


Answer: Okay, so after some effort, I finally managed to get a simple **"hello
world"** example working for setuptools. The Python documentation is usually
amazing, but I wish the documentation was better on this in particular.

I'm going to write a fairly detailed guide on how I achieved this, and I'll
assume no prior background on the reader on this topic. I hope this comes in
handy for others...

In order to get this example set up, we'll be creating a package (actually two
of them, one for the data files). This is the directory structure we'll end up
with:

    
    
    test-setuptools/
    |-- helloworld/
    |-- --- hello.py
    |-- --- images/
    |-- --- --- hello.gif
    |-- --- --- __init__.py
    |-- --- __init__.py
    |-- MANIFEST.in
    |-- README.txt
    |-- setup.py
    

Here are the steps:

  1. Create the `helloworld` package.  
1.1 Create the `helloworld/` folder as shown in the directory structure above.

1.2 Add a blank file called `__init__.py` in the `helloworld/` folder.  
If you don't add it, the package won't be recognized (run `touch __init__.py`
to create the file on linux/mac machines). If you want some code to be
executed every time the package is imported, include it in the `__init__.py`
file.

1.3 Create the the `hello.py` script file to demonstrate the package
functionality.

Here is the code for `hello.py`:

    
    
    import os
    
    """
    Open additional data files using the absolute path,
    otherwise it doesn't always find the file.
    """
    # The absolute path of the directoy for this file:
    _ROOT = os.path.abspath(os.path.dirname(__file__))
    
    class Hello(object):
        def say_hello(self):
            return "Hello, World!"
        def open_image(self):
            print("Reading image.gif contents:")
    
            # Get the absolute path of the image's relative path:
            absolute_image_path = os.path.join(_ROOT, 'images/hello.gif')
    
            with open(absolute_image_path, "r") as f:
                for line in f:
                    print(line)
    

  1. 1.4 Create the `images/` folder inside the `helloworld/` folder.  
Make another blank `__init__.py` file, because this folder will also be a
package.

1.5 Create the `hello.gif` file inside the `images/` folder.  
This file won't be an actual gif file. Instead, add plain text just to
demonstrate that non-script files can be added and read.

I added the following code in `hello.gif`:

    
    
    This should be the data inside hello.gif...
    ...but this is just to demonstrate setuptools,
    so it's a dummy gif containing plain text
    

  1. 1.6 Test your package  
Run `python` from the `test-setuptools` folder, which will open the python
interpreter.  
Type `import helloworld.hello` to import the `hello.py` script in the
`helloworld` package. The import should be successful, indicating that you
successfully created a package. Make sure that the package in the `images/`
folder also works, by typing `import helloworld.images`  
Try instantiating the object that we wrote in `hello.py`. Type the following
commands to make sure everything works as expected:  
`hey = helloworld.hello.Hello()`  
`hey.say_hello()`  
`hey.open_image()`

  2. Create the `setup.py` file and the remaining files.  
2.1 Create a simple `README.txt` file. Mine just has the text: `Hello, World!
Readme` inside.

2.2 Create a `MANIFEST.in` file with the following contents:  
`include helloworld/images/hello.gif`  
.  
This is very **important** because it tells setuptools to include the
additional data in the source distribution (which we'll generate in a later
step). Without this, you won't be able to install additional, non `.py` data
to your package. See
[this](http://docs.python.org/2/distutils/sourcedist.html#manifest-template)
for additional details and commands.

2.3 Create the `setup.py` file (see the code below).  
The most important attributes are `packages`, `include_package_data`, and
`package_data`.  
.  
The `packages` attribute contains a list of the packages you want to include
for setuptools. We want to include both the `helloworld` package and the
`helloworld.images` package that contains our additional data `hello.gif`.  
You can make setuptools automatically find these by adding the `from
setuptools import find_packages` import and running the imported
`find_packages()` function. Run the interpreter from the `test-setuptools`
folder and test this command to see which packages are found.  
.  
The `package_data` attribute tells setuptools to include additional data. It's
this command, the `helloworld.images` package, and the `MANIFEST.in` file
which allow you to install additional data.  
The `'helloworld.images' : ['hello.gif']` key/value pair tells setuptools to
include `hello.gif` inside the `helloworld.images` package if it exists. You
can also say `'' : ['*.gif']` to include any .gif file in any of the included
packages.  
The `include_package_data` attribute set to `True` is also necessary for this
to work.  
You can include additional metadata for the package like I have (I think an
`author` is necessary). It's a good idea to add classifiers. Additional info
can be found [here](http://docs.python.org/2/distutils/setupscript.html).

Here is the entire `setup.py` code:

    
    
    from setuptools import setup
    
    setup(
        name='helloworld',
        version='0.1',
        license='BSD',
        author='gyeh',
        author_email='hello@world.com',
        url='http://www.hello.com',
        long_description="README.txt",
        packages=['helloworld', 'helloworld.images'],
        include_package_data=True,
        package_data={'helloworld.images' : ['hello.gif']},
        description="Hello World testing setuptools",
    )
    

.  
3\. Install and test your package with setuptools.

    
    
    3.1 Create the source distribution  
    

Run `python setup.py sdist` from the `test-setuptools/` folder to generate the
source distribution.  
This will create a `dist/` folder containing your package, and a
`helloworld.egg-info/` folder containing metadata such as `SOURCE.txt`.  
Check `SOURCE.txt` to see if your the `hello.gif` image file is included
there.  
Open the `.tar.gz` file under the `dist/` folder. You should see all of the
files described in the directory structure we made earlier, including
`hello.gif` and `hello.py`.

    
    
    3.2 Install the distribution  
    

Install the .tar.gz distribution file by running `pip install --user
helloworld-0.1.tar.gz` from the `dist/` folder.  
Check that the package was successfully installed by running `pip list`. The
package `helloworld` should be there.

That's it! now you should be able to test your package under any folder. Open
up the interpreter in any folder except `test-setuptools`, and try importing
the package using `import helloworld.hello`. It should work. Then try the
commands to instantiate the object and open the image file using the
`hey.open_image()` command again. It should still work!

You can view exactly which files were installed by pip, and where they are, by
uninstalling the package. Mine looked like this:

    
    
    [gyeh@gyeh package]$ pip uninstall helloworld
    Uninstalling helloworld:
      /home/gyeh/.local/lib/python3.3/site-packages/helloworld-0.1-py3.3.egg-info
      /home/gyeh/.local/lib/python3.3/site-packages/helloworld/__init__.py
      /home/gyeh/.local/lib/python3.3/site-packages/helloworld/__pycache__/__init__.cpython-33.pyc
      /home/gyeh/.local/lib/python3.3/site-packages/helloworld/__pycache__/hello.cpython-33.pyc
      /home/gyeh/.local/lib/python3.3/site-packages/helloworld/hello.py
      /home/gyeh/.local/lib/python3.3/site-packages/helloworld/images/__init__.py
      /home/gyeh/.local/lib/python3.3/site-packages/helloworld/images/__pycache__/__init__.cpython-33.pyc
      /home/gyeh/.local/lib/python3.3/site-packages/helloworld/images/hello.gif
    Proceed (y/n)? y
      Successfully uninstalled helloworld
    

As you can see, it successfully installed the additional data file
`hello.gif`, and because we converted the relative path to an absolute path in
`hello.py`, it can read the file just fine.  
You can then share this package on PyPI for the rest of the world to use! The
instructions on uploading to PyPI are fairly straightforward and can be found
[here](http://docs.python.org/3/distutils/packageindex.html) and
[here](http://docs.python.org/3.1/distutils/uploading.html).

Once it's online in PyPI, people can search for your package using `pip
search`. Or alternatively, running `pip install --user [package-name]` will
tell pip to search the online PyPI directory for that package name. If it
exists, it will install it.

You can run that command for any python package that in PyPI for an easy
install so you aren't mucking around with build files.

I hope this saves people a bunch of headaches.

