Virtualenv Tutorial

Here is a newer version of this virtualenv tutorial.

What is Virtualenv

Virtualenv is a great piece of software. It allows to create some virtual environments that have many different versions of Python across the whole system and having different sets of libraries.

Tools Versions

While creating this tutorial I used following tools and versions:

  • virtualenv – 1.4.8
  • Python – 2.6.5
  • Ubuntu 10.04 64 bit
  • All console programs run with UTF-8

Do I Need Virtual Environment?

As usually: that depends. Normally you can create and use Python programs without that but using virtualenv can help a lot.

Such virtual environment provides many possibilities such as:

  • On production servers it allows to run applications created for different Python versions.
  • On testing servers it allows to perform many tests including:
    • Testing the installer script if that really installs all necessary libraries with checking their versions.
    • Testing applications using different set of libraries.
    • Checking if upgrade of a library won’t cause errors.

Basic Steps with Virtualenv

Installation

Installation is very easy. The best way is to perform system wide installation so all users can create and use virtualenv program, this can be useful later. For such an install you have to do it as the main administrator, that’s why there is the ‘sudo’ part of the command. I use Ubuntu on my laptop, and I will use the sudo command, on other systems that could be done some other way. The command for installing is simple:

sudo easy_install virtualenv

Successful command execution should print on the console something like this:

sudo easy_install virtualenv
Searching for virtualenv
Best match: virtualenv 1.4.8
Processing virtualenv-1.4.8-py2.6.egg
Adding virtualenv 1.4.8 to easy-install.pth file
Installing virtualenv script to /usr/local/bin

Using /usr/local/lib/python2.6/dist-packages/virtualenv-1.4.8-py2.6.egg
Processing dependencies for virtualenv
Finished processing dependencies for virtualenv

Creating the First Virtual Environment

Creating the virtual environment is very easy. First of all choose the directory that you want to create the environment in. The best location would be your home directory. Let’s create there a special directory where all different environments will be placed. The directory will be named ‘virt_env’. For creating this directory I’ll use the normal mkdir command:

mkdir virt_env

Now let’s create the first virtual environment inside this directory. Normally that’s done using the command:

virtualenv virt_env/virt1

but first let’s talk a bit about the most useful parameters that we can pass there.

Possible Parameters for Virtualenv
  • –help or -hStandard unix parameter used for getting some information about program. I’ve never found any program that didn’t follow that convention so it is good to remember that.
  • –verbose or -vThe program prints a lot more information than usually, that can be useful for solving some problems.
  • –quiet or -qOpposite to –verbose, the program prints as less information as possible.
  • –clearIn case you want to reinstall virtualenv in a directory where is another installation. Using this option causes deleting previous installation and installing new one, so you have a clean reinstall.
  • –versionPrints only the program version. For me currently that prints 1.4.8.
  • –no-site-packagesAll packages installed using easy_install are global. It means that all users have access to them and can use them. Sometimes it can be useful but now I want to have a pure virtualized environment regardless what I have installed in the system and in other virtual environments. Using this option allows to create what I want – the new environment won’t use any of the system python packages. A pure clean directory without any preinstalled packages.
How to Create The Virtual Environment for Python

I change now the command shown earlier by adding some parameters. I don’t want to use any preinstalled packages from my operating system, so the command looks like this:

virtualenv virt_env/virt1 --no-site-packages

Output for the command should be like this:

New python executable in virt_env/virt1/bin/python
Installing setuptools............done.

Output using the –verbose parameter is a little bit messy, so don’t use that normally, this is useful only to debug some internals when you want to know how it works or when it doesn’t work:

virtualenv virt_env/virt1 --no-site-packages --verbose

Creating virt_env/virt1/lib/python2.6
Symlinking Python bootstrap modules
  Symlinking virt_env/virt1/lib/python2.6/re.pyc
  Symlinking virt_env/virt1/lib/python2.6/codecs.pyc
  Symlinking virt_env/virt1/lib/python2.6/posixpath.pyo
  Symlinking virt_env/virt1/lib/python2.6/re.py
  Symlinking virt_env/virt1/lib/python2.6/stat.py
  Symlinking virt_env/virt1/lib/python2.6/UserDict.pyc
  Symlinking virt_env/virt1/lib/python2.6/posixpath.pyc
  Symlinking virt_env/virt1/lib/python2.6/re.pyo
  Symlinking virt_env/virt1/lib/python2.6/sre_constants.pyo
  Symlinking virt_env/virt1/lib/python2.6/sre.pyc
  Symlinking virt_env/virt1/lib/python2.6/sre_constants.py
  Symlinking virt_env/virt1/lib/python2.6/genericpath.pyc
  Symlinking virt_env/virt1/lib/python2.6/sre_compile.pyo
  Symlinking virt_env/virt1/lib/python2.6/posixpath.py
  Symlinking virt_env/virt1/lib/python2.6/lib-dynload
  Symlinking virt_env/virt1/lib/python2.6/copy_reg.pyc
  Symlinking virt_env/virt1/lib/python2.6/codecs.py
  Symlinking virt_env/virt1/lib/python2.6/copy_reg.py
  Symlinking virt_env/virt1/lib/python2.6/genericpath.pyo
  Symlinking virt_env/virt1/lib/python2.6/abc.pyo
  Symlinking virt_env/virt1/lib/python2.6/warnings.pyo
  Symlinking virt_env/virt1/lib/python2.6/stat.pyc
  Symlinking virt_env/virt1/lib/python2.6/sre_parse.py
  Symlinking virt_env/virt1/lib/python2.6/fnmatch.py
  Symlinking virt_env/virt1/lib/python2.6/fnmatch.pyc
  Symlinking virt_env/virt1/lib/python2.6/_abcoll.pyc
  Symlinking virt_env/virt1/lib/python2.6/os.py
  Symlinking virt_env/virt1/lib/python2.6/os.pyo
  Symlinking virt_env/virt1/lib/python2.6/encodings
  Symlinking virt_env/virt1/lib/python2.6/warnings.py
  Symlinking virt_env/virt1/lib/python2.6/UserDict.pyo
  Symlinking virt_env/virt1/lib/python2.6/abc.pyc
  Symlinking virt_env/virt1/lib/python2.6/locale.pyc
  Symlinking virt_env/virt1/lib/python2.6/sre_parse.pyc
  Symlinking virt_env/virt1/lib/python2.6/_abcoll.py
  Symlinking virt_env/virt1/lib/python2.6/types.pyo
  Symlinking virt_env/virt1/lib/python2.6/config
  Symlinking virt_env/virt1/lib/python2.6/codecs.pyo
  Symlinking virt_env/virt1/lib/python2.6/stat.pyo
  Symlinking virt_env/virt1/lib/python2.6/locale.pyo
  Symlinking virt_env/virt1/lib/python2.6/ntpath.pyc
  Symlinking virt_env/virt1/lib/python2.6/copy_reg.pyo
  Symlinking virt_env/virt1/lib/python2.6/sre_constants.pyc
  Symlinking virt_env/virt1/lib/python2.6/genericpath.py
  Symlinking virt_env/virt1/lib/python2.6/types.py
  Symlinking virt_env/virt1/lib/python2.6/sre_compile.pyc
  Symlinking virt_env/virt1/lib/python2.6/ntpath.py
  Symlinking virt_env/virt1/lib/python2.6/os.pyc
  Symlinking virt_env/virt1/lib/python2.6/_abcoll.pyo
  Symlinking virt_env/virt1/lib/python2.6/sre_compile.py
  Symlinking virt_env/virt1/lib/python2.6/fnmatch.pyo
  Symlinking virt_env/virt1/lib/python2.6/linecache.pyc
  Symlinking virt_env/virt1/lib/python2.6/types.pyc
  Symlinking virt_env/virt1/lib/python2.6/sre.py
  Symlinking virt_env/virt1/lib/python2.6/abc.py
  Symlinking virt_env/virt1/lib/python2.6/warnings.pyc
  Symlinking virt_env/virt1/lib/python2.6/UserDict.py
  Symlinking virt_env/virt1/lib/python2.6/sre_parse.pyo
  Symlinking virt_env/virt1/lib/python2.6/linecache.pyo
  Symlinking virt_env/virt1/lib/python2.6/linecache.py
  Symlinking virt_env/virt1/lib/python2.6/locale.py
Creating virt_env/virt1/lib/python2.6/site-packages
Writing virt_env/virt1/lib/python2.6/site.py
Writing virt_env/virt1/lib/python2.6/orig-prefix.txt
Writing virt_env/virt1/lib/python2.6/no-global-site-packages.txt
Creating parent directories for virt_env/virt1/include
Symlinking virt_env/virt1/include/python2.6
Creating virt_env/virt1/bin
New python executable in virt_env/virt1/bin/python
Changed mode of virt_env/virt1/bin/python to 0755
Testing executable with virt_env/virt1/bin/python -c "import sys; print sys.prefix"
Got sys.prefix result: '/home/virt/virt_env/virt1'
Creating virt_env/virt1/lib/python2.6/distutils
Writing virt_env/virt1/lib/python2.6/distutils/__init__.py
Writing virt_env/virt1/lib/python2.6/distutils/distutils.cfg
Using existing setuptools egg: /usr/local/lib/python2.6/dist-packages/virtualenv-1.4.8-py2.6.egg/virtualenv_support/setuptools-0.6c11-py2.6.egg
Installing setuptools..........
  Processing dependencies for setuptools==0.6c11
  Finished processing dependencies for setuptools==0.6c11
...Installing setuptools...done.
Installing pip-0.7.1.tar.gz
  Processing pip-0.7.1.tar.gz
  Running pip-0.7.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xgmRQJ/pip-0.7.1/egg-dist-tmp-N5lGfG
  warning: no previously-included files matching '*.txt' found under directory 'docs/_build'
  no previously-included directories found matching 'docs/_build/_sources'
  zip_safe flag not set; analyzing archive contents...
  pip.venv: module references __file__
  pip.basecommand: module references __file__
  pip.runner: module references __file__
  pip.vcs.__init__: module references __file__
  Adding pip 0.7.1 to easy-install.pth file
  Processing dependencies for pip==0.7.1
  Finished processing dependencies for pip==0.7.1
Writing virt_env/virt1/bin/activate
Writing virt_env/virt1/bin/activate_this.py

Using the Virtual Python Environment

Activating Environment

Using the virtual environment is pretty simple. First of all you have to define what environment you want to use. So far there is just one environment but later will be more. Let’s use this one environment. The environment directory is

virt_env/virt1/

the command for loading this environment is:

virt@ymon:~$ source virt_env/virt1/bin/activate

Successful execution changes the prompt, now it looks like this:

(virt1)virt@ymon:~$

The first part of the prompt is the name of the virtual environment so it is very easy to see which environment is used at this time. Of course you can change the prompt to the previous version but I wouldn’t recommend that as it can become very messy when using more environments.

Deactivating Current Environment

Current environment can be easily deactivated using command

(virt1)virt@ymon:~$ deactivate

that reverts the prompt to previous version and now it looks like this:

virt@ymon:~$

The deactivate command exists only when a virtual environment is active so don’t be surprised that there isn’t any when you outside the environment.

Installing Packages in Environment

Checking What is Installed

For listing all installed Python packages I will use yolk. This is a simple console program which can list installed packages.

Installation is quite simple:

sudo easy_install yolk

Usage is even simpler:

yolk -l

Running that command outside the virtual environment shows me just 114 packages so I will not paste the whole package list here.

Using yolk in the new environment needs installing that locally, so let’s install:

virt@ymon:~$ source virt_env/virt1/bin/activate
(virt1)virt@ymon:~$ easy_install yolk
Searching for yolk
Best match: yolk 0.4.1
Processing yolk-0.4.1-py2.6.egg
yolk 0.4.1 is already the active version in easy-install.pth
Installing yolk script to /home/virt/virt_env/virt1/bin

Using /home/virt/virt_env/virt1/lib/python2.6/site-packages/yolk-0.4.1-py2.6.egg
Processing dependencies for yolk
Finished processing dependencies for yolk
(virt1)virt@ymon:~$

Now I can use that locally. On the new virtual environment there isn’t much packages:

(virt1)virt@ymon:~$ yolk -l
Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
pip             - 0.7.1        - active
setuptools      - 0.6c11       - active
wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
yolk            - 0.4.1        - active

Installing Other Packages

That’s easy, I can install packages normally, just in the one environment, not touching any other or globally packages. Now I will install Pylons in the environment, but first let’s create other one to ensure that the packages are really installed just in this one.

(virt1)virt@ymon:~$ deactivate
virt@ymon:~$ virtualenv virt_env/virt2 --no-site-packages
New python executable in virt_env/virt2/bin/python
Installing setuptools............done.

OK, now there is another environment, let’s check what packages are installed there (of course after installing yolk like in the previous one):

(virt2)virt@ymon:~$ yolk -l
Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
pip             - 0.7.1        - active
setuptools      - 0.6c11       - active
wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
yolk            - 0.4.1        - active

Great, the list looks exactly like the previous one.

Now I just switch to the first environment and install Pylons.

(virt2)virt@ymon:~$ deactivate
virt@ymon:~$ source virt_env/virt1/bin/activate
(virt1)virt@ymon:~$ easy_install Pylons

This command prints a lot on the console as it is installing many other packages needed by Pylons. The last lines of the messages list looks like this:

Installed /home/virt/virt_env/virt1/lib/python2.6/site-packages/Pygments-1.3.1-py2.6.egg
Finished processing dependencies for Pylons
(virt1)virt@ymon:~$

And also let’s install SQLAlchemy library:

(virt1)virt@ymon:~$ easy_install SqlAlchemy

And now let’s check the list of installed packages:

(virt1)virt@ymon:~$ yolk -l
Beaker          - 1.5.3        - active
FormEncode      - 1.2.2        - active
Mako            - 0.3.2        - active
Paste           - 1.7.3.1      - active
PasteDeploy     - 1.3.3        - active
PasteScript     - 1.7.3        - active
Pygments        - 1.3.1        - active
Pylons          - 0.10rc1      - active
Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
Routes          - 1.12.1       - active
SQLAlchemy      - 0.6.0        - active
Tempita         - 0.4          - active
WebError        - 0.10.2       - active
WebHelpers      - 1.0b6        - active
WebOb           - 0.9.8        - active
WebTest         - 1.2.1        - active
decorator       - 3.1.2        - active
nose            - 0.11.3       - active
pip             - 0.7.1        - active
setuptools      - 0.6c11       - active
simplejson      - 2.1.1        - active
wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
yolk            - 0.4.1        - active

As you can see there is installed the version 0.6.0 of SQLAlchemy library.

Great, it just works but what about the second environment, does it have those packages also installed?

(virt1)virt@ymon:~$ deactivate
virt@ymon:~$ source virt_env/virt2/bin/activate
(virt2)virt@ymon:~$ yolk -l
Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
pip             - 0.7.1        - active
setuptools      - 0.6c11       - active
wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
yolk            - 0.4.1        - active

It seems that everything is OK, so let’s install there Pylons and check the list of installed packages:

(virt1)virt@ymon:~$ deactivate
virt@ymon:~$ source virt_env/virt2/bin/activate
(virt2)virt@ymon:~$ easy_install Pylons
[... here a lot of messages ...]
Processing dependencies for Pylons
Finished processing dependencies for Pylons
(virt2)virt@ymon:~$ yolk -l
Beaker          - 1.5.3        - active
FormEncode      - 1.2.2        - active
Mako            - 0.3.2        - active
Paste           - 1.7.3.1      - active
PasteDeploy     - 1.3.3        - active
PasteScript     - 1.7.3        - active
Pygments        - 1.3.1        - active
Pylons          - 0.10rc1      - active
Python          - 2.6.5        - active development (/usr/lib/python2.6/lib-dynload)
Routes          - 1.12.1       - active
Tempita         - 0.4          - active
WebError        - 0.10.2       - active
WebHelpers      - 1.0b6        - active
WebOb           - 0.9.8        - active
WebTest         - 1.2.1        - active
decorator       - 3.1.2        - active
nose            - 0.11.3       - active
pip             - 0.7.1        - active
setuptools      - 0.6c11       - active
simplejson      - 2.1.1        - active
wsgiref         - 0.1.2        - active development (/usr/lib/python2.6)
yolk            - 0.4.1        - active

Great, but there is no SQLAlchemy. So let’s install that but using the previous version, that means installing the version 0.5.0.

(virt2)virt@ymon:~$ easy_install "SQLAlchemy==0.5.0"
Searching for SQLAlchemy==0.5.0
Reading http://pypi.python.org/simple/SQLAlchemy/
Reading http://www.sqlalchemy.org
Best match: SQLAlchemy 0.5.0
Downloading http://pypi.python.org/packages/source/S/SQLAlchemy/SQLAlchemy-0.5.0.tar.gz##md5=df49f403b2db3c54aace64aebe26cf90
Processing SQLAlchemy-0.5.0.tar.gz
Running SQLAlchemy-0.5.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-T1eI2f/SQLAlchemy-0.5.0/egg-dist-tmp-jnaNHq
no previously-included directories found matching 'doc/build/output'
zip_safe flag not set; analyzing archive contents...
sqlalchemy.databases.mysql: module MAY be using inspect.stack
Adding SQLAlchemy 0.5.0 to easy-install.pth file

Installed /home/virt/virt_env/virt2/lib/python2.6/site-packages/SQLAlchemy-0.5.0-py2.6.egg
Processing dependencies for SQLAlchemy==0.5.0
Finished processing dependencies for SQLAlchemy==0.5.0

Looks fine, lets check the installed version:

(virt2)virt@ymon:~$ yolk -l | grep SQLAlchemy
SQLAlchemy      - 0.5.0        - active

Summary

Now I’ve got two Python environments in directories:

virt_env/virt1/
virt_env/virt2/

in both there is the newest version of Pylons installed.

In the first there is SQLAlchemy 0.6.0
In the second there is SQLAlchemy 0.5.0.

Now that’s easy to test the same Pylons application how it behaves on both versions of SQLAlchemy without any changes in configuration.

Here is a newer version of this virtualenv tutorial.

30 responses

  1. Thank you for this tutorial, definitely easy to follow and understand. Finally I got a quick intro to VirtualEnv.
    What about “Add description of using different Python versions”?

  2. @Giuseppe, yes, I’m thinking about that, as well as about writing the same tutorial for windows. I hope to finish that very soon.

  3. Thanks! I love it when someone manages to explain how to use complex software, when it’s simple after all. You’ve got a talent, Simon!

  4. Hi Joey,
    thanks for the nice words. The tutorial is a little bit outdated, but the brand new shiny version is on its way and I will publish that soon.

  5. Hey Simon, Thanks for the tutorial! Very clean and right to the point. When you update it are you going to add virtualwrapper?

  6. Hi Jamie,
    It will be published soon… however without virtualwrapper at this time. Next version should be with virtualwrapper and I’m also working on the Windows version (as it is more complicated there).
    By soon I think in the near future, but I cannot give exact data (I’ve got too many things to do right now).

  7. Hi Simon,

    I am not able to isolate my virtual environment. I created the two virt1 and virt2.
    if I install easy_install yolk. Then it shows the same modules of python as are without virtualenv.I have checked it many a times.I am using python -2.6,virtualenv-1.7.1.2 and ubuntu10.10

  8. Thanks, I was confused about pythons virtual env setup but after reading your article I was able to set up on my linux box. Thanks again.

  9. I get the same issues as Nishant above. Any pointers?

    “I am not able to isolate my virtual environment. I created the two virt1 and virt2.
    if I install easy_install yolk. Then it shows the same modules of python as are without virtualenv.I have checked it many a times.I am using python -2.6,virtualenv-1.7.1.2 and ubuntu10.10″

  10. To elaborate, any installed packages whilst my virtualenv is active are installed to my system python modules. I wondered if this is to do with my PYTHONPATH but there is no pythonpath in my bash profile and everything works fine with my system python modules.

  11. @Luke and @Nishant @Simon

    I got a similar issue but it only happened when I executed “yolk -l” on the second environment without actually having it installed there. It somehow was picking up my global Yolk and printing the full list of installed packages, therefore making both environments look the same.

    Solution, make sure Yolk is installed in both environments independently and then run “yolk -l” to check the installed list of packages

  12. Thanks for the tutorial! I ended up here from a stackoverflow post and you solved most of my issues. Just one thing I’d like to add: the activate command didn’t work in my case (i’m using debian wheezy), and I just found out that I needed to prepend the command with the “source” command. Now it works just fine.

Leave a Reply

Your email address will not be published. Required fields are marked *