An Exploration of Python Dependencies (Part I)

This is intended as an absolute beginner Python programmer’s guide to the PYTHONPATH and importing your own modules to your projects easily. This should work on most Unix-like (*nix) machines.

I struggled for a while when I first started using Linux and programming Python to figure out exactly how the Python path worked and how to add my own custom modules to the path. Inevitably I ended up either starting the shell I was programming from in the directory I wanted to import modules from (I never really understood why this worked), or going through the rather lengthy (programmers love elegant shortcuts!) process of appending directories to my PYTHONPATH. I’d say I did this for a really long time, but the process I went through was valuable because we learn through practise, not merely being told how something works. I encourage you to follow along and attempt these examples on your own machine.

Please note that
———————————————————————————————————————-
$:
———————————————————————————————————————-
is used to indicate your system shell (I use bash), and
———————————————————————————————————————-
>>>
———————————————————————————————————————-
indicates the Python interpreter prompt.

Here’s how you can find and set your python path:
———————————————————————————————————————-
$:python
>>> _
>>> import sys
>>> print sys.path
———————————————————————————————————————-

On my machine, I get the following output:
———————————————————————————————————————-
['', '/usr/local/lib/python2.7/dist-packages/pip-1.0.1-py2.7.egg', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/pymodules/python2.7/gtk-2.0', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/pymodules/python2.7/ubuntuone-storage-protocol', '/usr/lib/pymodules/python2.7/ubuntuone-client', '/usr/lib/pymodules/python2.7/ubuntuone-control-panel', '/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode']
———————————————————————————————————————-

As you can see, my sys.path is quite bare (the only modules I have installed are the ones required for my Ubuntu installation), because I use virtualenv to manage my dependencies — more about that in a future post.

The data structure above is called a list, identified by the square brackets. We have also solved the mystery of why you can import from the current directory; the first entry, the empty string, tells the interpreter to look in the current directory.

So, to add a directory to the Python path, you can do:
———————————————————————————————————————-
>>> sys.path.append('<path_to_your_module_directory>')
———————————————————————————————————————-

Remember to make sure that <path_to_your_module_directory> has an empty file called __init__.py in it. This lets the Python interpreter know that the directory is a Python module.

As a useful aside, to create an empty file quickly from the terminal:
———————————————————————————————————————-
$: touch __init__.py
———————————————————————————————————————-

Say you appended '/home/nikhil/src/mymodules/' to sys.path. The directory structure could look something like this:
———————————————————————————————————————-
mymodules/
__init__.py
mytools.py
———————————————————————————————————————-

You could now do something like this:
———————————————————————————————————————-
>>> from mytools import *
———————————————————————————————————————-
to use all the functions, variables and classes in mytools.

However, if you restart the shell (CTRL+D to quit), and check out the Python path, you will notice that the sys.path that you set is not persistent across sessions. To remedy this for the time being, you could add the following to your Python files to ensure you are including the required modules on your path:
———————————————————————————————————————-
import sys
sys.path.append('<path_to_your_module_directory>')
———————————————————————————————————————-

Now you have a rudimentary solution as to how to include directories to the python path. You could even use this for libraries that you download.

In the next post, we will be looking at a handy bash script to automate this process, and link and unlink directories quickly.