Virtualenv Tutorial

Fri, Apr 30, 2010

by Szymon Lipiński

tags: python virtualenv

Here is a newer version of this virtualenv tutorial.

What Virtualenv Is

Virtualenv is a great piece of software. It allows to create virtual environments. Each of them can have a different version of Python and different sets of libraries.

While creating this tutorial I used following tools and versions:

virtualenv - 1.4.8
Python - 2.6.5
Ubuntu 10.04 64 bit
All console programs run with UTF-8

Do I Need a Virtual Environment?

As usually: it depends. Normally you can create and use Python programs without that, but using virtual environments can help a lot.

Such a virtual environment provides many possibilities such as:

On production servers it allows to run applications created for different Python versions.
On testing servers it allows to perform many tests including:
- Testing the installer script if that really installs all necessary libraries with checking their versions.
- Testing applications using different set of libraries.
- Checking if upgrade of a library won’t cause errors.

Basic Steps with Virtualenv

Installation

Installation is very easy. The best way is to perform a system wide installation so all users can create, and use the virtualenv. This must be done from an administrator account. I use Ubuntu on my laptop and I will use the sudo command, on other systems that could be done some other way.

The command for installing is simple:

    sudo easy_install virtualenv

A successful command execution should print on the console something like this:

    sudo easy_install virtualenv
    Searching for virtualenv
    Best match: virtualenv 1.4.8
    Processing virtualenv-1.4.8-py2.6.egg
    Adding virtualenv 1.4.8 to easy-install.pth file
    Installing virtualenv script to /usr/local/bin

    Using /usr/local/lib/python2.6/dist-packages/virtualenv-1.4.8-py2.6.egg
    Processing dependencies for virtualenv
    Finished processing dependencies for virtualenv

Creating the First Virtual Environment

Creating the virtual environment is very easy. First of all choose the directory where you want to create it. The best location would be your home directory.

Let’s create there a special directory where all different environments will be placed. The directory will be named virt_env. For creating this directory I’ll use the normal mkdir command:

    mkdir virt_env

Now let’s create the first virtual environment inside this directory. Normally that’s done using the command:

    virtualenv virt_env/virt1

but first let’s talk a bit about the most useful parameters that we can pass there.

Possible Parameters for Virtualenv

--help or -h – A standard unix parameter used for getting information about program. I’ve never found any program that didn’t follow that convention so it is good to remember that.
--verbose or -v – The program prints a lot more information than usually, that can be useful for solving some problems.
--quiet or -q – Opposite to the --verbose, the program prints as less information as possible.
--clear – In case you want to reinstall the virtualenv in a directory where is another installation. Using this option causes deleting previous installation, and installing new one, so you have a clean reinstall.
--version – Prints only the program version. For me currently that prints 1.4.8.
--no-site-packages – All packages installed using easy_install are global. It means that all users have access to them, and can use them. Sometimes it can be useful but now I want to have a pure virtualized environment regardless what I have installed in the system and in other virtual environments. Using this option allows to create what I want - the new environment won’t use any of the system python packages. A pure clean directory without any preinstalled things.

How to Create The Virtual Environment for Python

I will change now the command shown earlier by adding some parameters. I don’t want to use any preinstalled packages from my operating system, so the command looks like this:

    virtualenv virt_env/virt1 --no-site-packages

Output for the command should be like this:

    New python executable in virt_env/virt1/bin/python
    Installing setuptools............done.

Output using the --verbose parameter is a little bit messy, so don’t use it normally. This is useful only to debug some internals when you want to know how it works or when it doesn’t work:

    virtualenv virt_env/virt1 --no-site-packages --verbose

    Creating virt_env/virt1/lib/python2.6
    Symlinking Python bootstrap modules
      Symlinking virt_env/virt1/lib/python2.6/re.pyc
      Symlinking virt_env/virt1/lib/python2.6/codecs.pyc
      Symlinking virt_env/virt1/lib/python2.6/posixpath.pyo
      Symlinking virt_env/virt1/lib/python2.6/re.py
      Symlinking virt_env/virt1/lib/python2.6/stat.py
      Symlinking virt_env/virt1/lib/python2.6/UserDict.pyc
      Symlinking virt_env/virt1/lib/python2.6/posixpath.pyc
      Symlinking virt_env/virt1/lib/python2.6/re.pyo
      Symlinking virt_env/virt1/lib/python2.6/sre_constants.pyo
      Symlinking virt_env/virt1/lib/python2.6/sre.pyc
      Symlinking virt_env/virt1/lib/python2.6/sre_constants.py
      Symlinking virt_env/virt1/lib/python2.6/genericpath.pyc
      Symlinking virt_env/virt1/lib/python2.6/sre_compile.pyo
      Symlinking virt_env/virt1/lib/python2.6/posixpath.py
      Symlinking virt_env/virt1/lib/python2.6/lib-dynload
      Symlinking virt_env/virt1/lib/python2.6/copy_reg.pyc
      Symlinking virt_env/virt1/lib/python2.6/codecs.py
      Symlinking virt_env/virt1/lib/python2.6/copy_reg.py
      Symlinking virt_env/virt1/lib/python2.6/genericpath.pyo
      Symlinking virt_env/virt1/lib/python2.6/abc.pyo
      Symlinking virt_env/virt1/lib/python2.6/warnings.pyo
      Symlinking virt_env/virt1/lib/python2.6/stat.pyc
      Symlinking virt_env/virt1/lib/python2.6/sre_parse.py
      Symlinking virt_env/virt1/lib/python2.6/fnmatch.py
      Symlinking virt_env/virt1/lib/python2.6/fnmatch.pyc
      Symlinking virt_env/virt1/lib/python2.6/_abcoll.pyc
      Symlinking virt_env/virt1/lib/python2.6/os.py
      Symlinking virt_env/virt1/lib/python2.6/os.pyo
      Symlinking virt_env/virt1/lib/python2.6/encodings
      Symlinking virt_env/virt1/lib/python2.6/warnings.py
      Symlinking virt_env/virt1/lib/python2.6/UserDict.pyo
      Symlinking virt_env/virt1/lib/python2.6/abc.pyc
      Symlinking virt_env/virt1/lib/python2.6/locale.pyc
      Symlinking virt_env/virt1/lib/python2.6/sre_parse.pyc
      Symlinking virt_env/virt1/lib/python2.6/_abcoll.py
      Symlinking virt_env/virt1/lib/python2.6/types.pyo
      Symlinking virt_env/virt1/lib/python2.6/config
      Symlinking virt_env/virt1/lib/python2.6/codecs.pyo
      Symlinking virt_env/virt1/lib/python2.6/stat.pyo
      Symlinking virt_env/virt1/lib/python2.6/locale.pyo
      Symlinking virt_env/virt1/lib/python2.6/ntpath.pyc
      Symlinking virt_env/virt1/lib/python2.6/copy_reg.pyo
      Symlinking virt_env/virt1/lib/python2.6/sre_constants.pyc
      Symlinking virt_env/virt1/lib/python2.6/genericpath.py
      Symlinking virt_env/virt1/lib/python2.6/types.py
      Symlinking virt_env/virt1/lib/python2.6/sre_compile.pyc
      Symlinking virt_env/virt1/lib/python2.6/ntpath.py
      Symlinking virt_env/virt1/lib/python2.6/os.pyc
      Symlinking virt_env/virt1/lib/python2.6/_abcoll.pyo
      Symlinking virt_env/virt1/lib/python2.6/sre_compile.py
      Symlinking virt_env/virt1/lib/python2.6/fnmatch.pyo
      Symlinking virt_env/virt1/lib/python2.6/linecache.pyc
      Symlinking virt_env/virt1/lib/python2.6/types.pyc
      Symlinking virt_env/virt1/lib/python2.6/sre.py
      Symlinking virt_env/virt1/lib/python2.6/abc.py
      Symlinking virt_env/virt1/lib/python2.6/warnings.pyc
      Symlinking virt_env/virt1/lib/python2.6/UserDict.py
      Symlinking virt_env/virt1/lib/python2.6/sre_parse.pyo
      Symlinking virt_env/virt1/lib/python2.6/linecache.pyo
      Symlinking virt_env/virt1/lib/python2.6/linecache.py
      Symlinking virt_env/virt1/lib/python2.6/locale.py
    Creating virt_env/virt1/lib/python2.6/site-packages
    Writing virt_env/virt1/lib/python2.6/site.py
    Writing virt_env/virt1/lib/python2.6/orig-prefix.txt
    Writing virt_env/virt1/lib/python2.6/no-global-site-packages.txt
    Creating parent directories for virt_env/virt1/include
    Symlinking virt_env/virt1/include/python2.6
    Creating virt_env/virt1/bin
    New python executable in virt_env/virt1/bin/python
    Changed mode of virt_env/virt1/bin/python to 0755
    Testing executable with virt_env/virt1/bin/python -c "import sys; print sys.prefix"
    Got sys.prefix result: '/home/virt/virt_env/virt1'
    Creating virt_env/virt1/lib/python2.6/distutils
    Writing virt_env/virt1/lib/python2.6/distutils/__init__.py
    Writing virt_env/virt1/lib/python2.6/distutils/distutils.cfg
    Using existing setuptools egg: /usr/local/lib/python2.6/dist-packages/virtualenv-1.4.8-py2.6.egg/virtualenv_support/setuptools-0.6c11-py2.6.egg
    Installing setuptools..........
      Processing dependencies for setuptools==0.6c11
      Finished processing dependencies for setuptools==0.6c11
    ...Installing setuptools...done.
    Installing pip-0.7.1.tar.gz
      Processing pip-0.7.1.tar.gz
      Running pip-0.7.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xgmRQJ/pip-0.7.1/egg-dist-tmp-N5lGfG
      warning: no previously-included files matching '*.txt' found under directory 'docs/_build'
      no previously-included directories found matching 'docs/_build/_sources'
      zip_safe flag not set; analyzing archive contents...
      pip.venv: module references __file__
      pip.basecommand: module references __file__
      pip.runner: module references __file__
      pip.vcs.__init__: module references __file__
      Adding pip 0.7.1 to easy-install.pth file
      Processing dependencies for pip==0.7.1
      Finished processing dependencies for pip==0.7.1
    Writing virt_env/virt1/bin/activate
    Writing virt_env/virt1/bin/activate_this.py

Using the Virtual Python Environment

Activating the Environment

Using the virtual environment is pretty simple. First of all you have to define what environment you want to use. So far there is just one environment but later will be more.

Let’s use this one environment. The environment directory is

    virt_env/virt1/

the command for loading this environment is:

    virt@ymon:~$ source virt_env/virt1/bin/activate

A successful execution changes the prompt, now it looks like this:

    (virt1)virt@ymon:~$

The first part of the prompt is the name of the virtual environment so it is very easy to see which environment is currently used. Of course you can change the prompt to the previous version but I wouldn’t recommend that as it can become very messy when using more environments.

Deactivating The Current Environment

Current environment can be easily deactivated using command

    (virt1)virt@ymon:~$ deactivate

that reverts the prompt to previous version and now it looks like this:

    virt@ymon:~$

The deactivate command exists only when a virtual environment is active, so don’t be surprised that there isn’t any when you outside the environment.

Installing Packages in Environment

Checking What is Installed

For listing all installed Python packages I will use yolk. This is a simple console program which can list installed packages.

Installation is quite simple:

    sudo easy_install yolk

Usage is even simpler:

    yolk -l

Running that command outside the virtual environment shows me just 114 packages so I will not paste the whole list here.

Using the yolk in the new environment needs installing that locally, so let’s install:

    virt@ymon:~$ source virt_env/virt1/bin/activate
    (virt1)virt@ymon:~$ easy_install yolk
    Searching for yolk
    Best match: yolk 0.4.1
    Processing yolk-0.4.1-py2.6.egg
    yolk 0.4.1 is already the active version in easy-install.pth
    Installing yolk script to /home/virt/virt_env/virt1/bin

    Using /home/virt/virt_env/virt1/lib/python2.6/site-packages/yolk-0.4.1-py2.6.egg
    Processing dependencies for yolk
    Finished processing dependencies for yolk
    (virt1)virt@ymon:~$

Now I can use that locally. On the new virtual environment there isn’t much packages:

    (virt1)virt@ymon:~$ yolk -l
    Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
    pip             - 0.7.1        - active
    setuptools      - 0.6c11       - active
    wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
    yolk            - 0.4.1        - active

Installing Other Packages

Now I can install all packages normally, just in the one environment. It will not touch any other packages from other environments, and even the ones globally packages.

Now I will install Pylons in the environment, but first let’s create other one to ensure that the packages are really installed just in this one.

    (virt1)virt@ymon:~$ deactivate
    virt@ymon:~$ virtualenv virt_env/virt2 --no-site-packages
    New python executable in virt_env/virt2/bin/python
    Installing setuptools............done.

OK, now there is another environment, let’s check what packages are installed there (of course after installing yolk like in the previous one):

    (virt2)virt@ymon:~$ yolk -l
    Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
    pip             - 0.7.1        - active
    setuptools      - 0.6c11       - active
    wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
    yolk            - 0.4.1        - active

Great, the list looks exactly like the previous one.

Now I just switch to the first environment and install Pylons.

    (virt2)virt@ymon:~$ deactivate
    virt@ymon:~$ source virt_env/virt1/bin/activate
    (virt1)virt@ymon:~$ easy_install Pylons

This command prints a lot in the console as it installs many other packages needed by Pylons. The last lines of the messages list look like this:

    Installed /home/virt/virt_env/virt1/lib/python2.6/site-packages/Pygments-1.3.1-py2.6.egg
    Finished processing dependencies for Pylons
    (virt1)virt@ymon:~$

And also let’s install SQLAlchemy library:

    (virt1)virt@ymon:~$ easy_install SqlAlchemy

And now let’s check the list of installed packages:

    (virt1)virt@ymon:~$ yolk -l
    Beaker          - 1.5.3        - active
    FormEncode      - 1.2.2        - active
    Mako            - 0.3.2        - active
    Paste           - 1.7.3.1      - active
    PasteDeploy     - 1.3.3        - active
    PasteScript     - 1.7.3        - active
    Pygments        - 1.3.1        - active
    Pylons          - 0.10rc1      - active
    Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
    Routes          - 1.12.1       - active
    SQLAlchemy      - 0.6.0        - active
    Tempita         - 0.4          - active
    WebError        - 0.10.2       - active
    WebHelpers      - 1.0b6        - active
    WebOb           - 0.9.8        - active
    WebTest         - 1.2.1        - active
    decorator       - 3.1.2        - active
    nose            - 0.11.3       - active
    pip             - 0.7.1        - active
    setuptools      - 0.6c11       - active
    simplejson      - 2.1.1        - active
    wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
    yolk            - 0.4.1        - active

As you can see there is installed the version 0.6.0 of SQLAlchemy library.

Great, it just works but what about the second environment? Does it have those packages also installed?

    (virt1)virt@ymon:~$ deactivate
    virt@ymon:~$ source virt_env/virt2/bin/activate
    (virt2)virt@ymon:~$ yolk -l
    Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
    pip             - 0.7.1        - active
    setuptools      - 0.6c11       - active
    wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
    yolk            - 0.4.1        - active

It seems that everything is OK, so let’s install there Pylons and check the list of installed packages:

    (virt1)virt@ymon:~$ deactivate
    virt@ymon:~$ source virt_env/virt2/bin/activate
    (virt2)virt@ymon:~$ easy_install Pylons
    [... here a lot of messages ...]
    Processing dependencies for Pylons
    Finished processing dependencies for Pylons
    (virt2)virt@ymon:~$ yolk -l
    Beaker          - 1.5.3        - active
    FormEncode      - 1.2.2        - active
    Mako            - 0.3.2        - active
    Paste           - 1.7.3.1      - active
    PasteDeploy     - 1.3.3        - active
    PasteScript     - 1.7.3        - active
    Pygments        - 1.3.1        - active
    Pylons          - 0.10rc1      - active
    Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
    Routes          - 1.12.1       - active
    Tempita         - 0.4          - active
    WebError        - 0.10.2       - active
    WebHelpers      - 1.0b6        - active
    WebOb           - 0.9.8        - active
    WebTest         - 1.2.1        - active
    decorator       - 3.1.2        - active
    nose            - 0.11.3       - active
    pip             - 0.7.1        - active
    setuptools      - 0.6c11       - active
    simplejson      - 2.1.1        - active
    wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
    yolk            - 0.4.1        - active

Great, but there is no SQLAlchemy. So let’s install that but using the previous version, that means installing the version 0.5.0.

    (virt2)virt@ymon:~$ easy_install "SQLAlchemy==0.5.0"
    Searching for SQLAlchemy==0.5.0
    Reading http://pypi.python.org/simple/SQLAlchemy/
    Reading http://www.sqlalchemy.org
    Best match: SQLAlchemy 0.5.0
    Downloading http://pypi.python.org/packages/source/S/SQLAlchemy/SQLAlchemy-0.5.0.tar.gz##md5=df49f403b2db3c54aace64aebe26cf90
    Processing SQLAlchemy-0.5.0.tar.gz
    Running SQLAlchemy-0.5.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-T1eI2f/SQLAlchemy-0.5.0/egg-dist-tmp-jnaNHq
    no previously-included directories found matching 'doc/build/output'
    zip_safe flag not set; analyzing archive contents...
    sqlalchemy.databases.mysql: module MAY be using inspect.stack
    Adding SQLAlchemy 0.5.0 to easy-install.pth file

    Installed /home/virt/virt_env/virt2/lib/python2.6/site-packages/SQLAlchemy-0.5.0-py2.6.egg
    Processing dependencies for SQLAlchemy==0.5.0
    Finished processing dependencies for SQLAlchemy==0.5.0

Looks fine, lets check the installed version:

    (virt2)virt@ymon:~$ yolk -l | grep SQLAlchemy
    SQLAlchemy      - 0.5.0        - active

The Summary

Now I’ve got two Python environments in directories:

    virt_env/virt1/
    virt_env/virt2/

in both there is a different version of Pylons installed.

    In the first there is SQLAlchemy 0.6.0
    In the second there is SQLAlchemy 0.5.0.

Now that’s easy to test the behaviour of the same Pylons application on both versions of SQLAlchemy without any changes in configuration.

Here is a newer version of this virtualenv tutorial.