18 October 2014

Virtual environments for real developments, playing with virtualenv

I don't know what you do but I usually develop multiple projects at the same time. Problem is that sometimes you need libraries for a project that are incompatible with those in another. That's why virtualenv exists in Python world.

Thanks to virtualenv you can keep all the Python versions, libraries, packages, and dependencies for a project separate from one another. It does it creating an isolated copy of python for your project directory without worry of affecting other projects. That is useful too if you have to develop and install libraries in a linux system where you have no sudo/root access to install anything.

Using virtualenv as part of your usual developments tools will keep you away future headaches.

First step about using virtualenv is to be sure about which version of python are you going to use in your project. Original virtualenv doesn't run right with Python 3, so if you use it you should keep yourself in Python 2.7. Fortunately, since 3.3 version Python includes it's own flavour of virtualenv built-in, called venv. Be aware that 3.3 version's venv install with no pip support but thankfully that support was added in Python 3.4. We are going to cover both of them in this article: first virtualenv and venv afterwards.

You can install virtualenv using your linux native package manager or python's pip installer. In Ubuntu you can install it through package manager doing:
dante@Camelot:~$ sudo aptitude install python-virtualenv

Using pip, should be enough with:
dante@Camelot:~$ pip install virtualenv

You can check which version you installed with:
dante@Camelot:~$ virtualenv --version

To use it, just navigate through console to your project directory an run following command:
dante@Camelot:~/project-directory$ virtualenv --no-site-packages env
New python executable in env/bin/python Installing setuptools, pip...done. dante@Camelot:~/project-directory$

That command will create a directory called env where binaries to run virtual environment are placed. The --no-site-packages flag truly isolates your work environment from the rest of your system as it does not include any packages or modules already installed on your system. That way, you have a completely isolated environment, free from any previously installed packages.

Before your work can start you should start your virtual environment running activate script:

dante@Camelot:~/project-directory$ source env/bin/activate (env)dante@Camelot:~/project-directory$
You know you are working in a virtualenv thanks to the directory name surrounded by parentheses to the left of the path in your command line. While you stay in  virtualenv all packages you install through pip will be stored in your virtual instance of python leaving your operating system python alone.

To leave virtualenv just type deactivate:

(env)dante@Camelot:~/project-directory$ deactivate dante@Camelot:~/project-directory$

You should create a virtualenv each time you start a new project.

Using venv in Python 3.4 is not so different. Nevertheless be aware that Ubuntu 14.04 comes with a broken version of venv. If you try to create a virtual environment with venv in Ubuntu 14.04, you'll get an error like this:

dante@Camelot:~/project-directory2$ pyvenv-3.4 env Error: Command '['/home/dante/project-directory2/env/bin/python3.4', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1 dante@Camelot:~/project-directory2$

The only way I've found to fix this problem is upgrade Ubuntu release to 14.10:

dante@Camelot:~/project-directory2$ sudo do-release-upgrade -d

Several hours later you'll get an upgraded Ubuntu 14.10 system. There you may find that venv needs to be installed from a python package:
dante@Camelot:~/project-directory2$ cat /etc/issue Ubuntu 14.10 \n \l dante@Camelot:~/project-directory2$ pyvenv-3.4 env El programa «pyvenv-3.4» no está instalado. Puede instalarlo escribiendo: sudo apt-get install python3.4-venv dante@Camelot:~/project-directory2$ sudo aptitude install python3.4-venv Se instalarán los siguiente paquetes NUEVOS: python3.4-venv 0 paquetes actualizados, 1 nuevos instalados, 0 para eliminar y 0 sin actualizar. Necesito descargar 1.438 kB de archivos. Después de desempaquetar se usarán 1.603 kB. Des: 1 http://es.archive.ubuntu.com/ubuntu/ utopic/universe python3.4-venv amd64 3.4.2-1 [1.438 kB] Descargados 1.438 kB en 0seg. (1.717 kB/s) Seleccionando el paquete python3.4-venv previamente no seleccionado. (Leyendo la base de datos ... 237114 ficheros o directorios instalados actualmente.) Preparing to unpack .../python3.4-venv_3.4.2-1_amd64.deb ... Unpacking python3.4-venv (3.4.2-1) ... Processing triggers for man-db (2.7.0.2-2) ... Configurando python3.4-venv (3.4.2-1) ... dante@Camelot:~/project-directory2$ pyvenv-3.4 env

That way venv runs with no errors:
dante@Camelot:~/project-directory2$ source env/bin/activate (env)dante@Camelot:~/project-directory2$

Once in your virtual environment you can install every package you need for your development using pip and without messing python libraries of your operating system. Modern linux distributions make heavy use of python applications, so it's a good practice to keep your operating system's python libraries clean with only what is really needed by your linux and mess with application specific libraries for your developments through their respectives virtual environments. For instance, you would use virtual environment if you needed to develop in Python 3 while keeping your operating system's default python interpreter in 2.7 version:

dante@Camelot:~/project-directory2$ source env/bin/activate (env) dante@Camelot:~/project-directory2$ python Python 3.4.2 (default, Oct 8 2014, 13:08:17) [GCC 4.9.1] on linux Type "help", "copyright", "credits" or "license" for more information. >>> exit() (env) dante@Camelot:~/project-directory2$ deactivate dante@Camelot:~/project-directory2$ python Python 2.7.8 (default, Oct 8 2014, 06:57:53) [GCC 4.9.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>

Don't be shy with virtual environments. You can use them in production environments, in fact they are what is actually recommended for production situations where you don't want an update of your operating system libraries break your running developed application dependencies.

12 comments:

  1. Replies
    1. I am glad that I saw this post. It is informative blog for us and we need this type of blog thanks for share this blog, Keep posting such instructional blogs and I am looking forward for your future posts. Python Projects for Students Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account. Project Center in Chennai

      Delete
  2. thanks for this valuable post about the dante lab
    such a awesome info bro,thanks for this post.if your looking for selenium courses or big data courses you can check out links below
    .Very well written article thanks for posting this amazing article with us keep growing and keep hustling
    Selenium course in chennai
    best selenium training institute in chennai
    best selenium training in chennai
    selenium training in chennai omr
    selenium training in omr
    big data course in chennai
    big data hadoop training in chennai
    big data analytics courses in chennai


    ReplyDelete
  3. I was very pleased to find this site. I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.

    Data Science Course

    ReplyDelete
  4. I feel very grateful that I read this. It is very helpful and very informative and I really learned a lot from it.
    Data Science Training Institute in Bangalore

    ReplyDelete

  5. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Definitely a great post. Hats off to you! The information that you have provided is very helpful.
    Best Data Science Courses in Bangalore

    ReplyDelete
  6. Two full thumbs up for this magnificant article of yours. I've really enjoyed reading this article today and I think this might be one of the best article that I've read yet. Please, keep this work going on in the same quality.

    Data Science Training

    ReplyDelete
  7. Going to graduate school was a positive decision for me. I enjoyed the coursework, the presentations, the fellow students, and the professors. And since my company reimbursed 100% of the tuition, the only cost that I had to pay on my own was for books and supplies. Otherwise, I received a free master’s degree. All that I had to invest was my time.
    Data Science Course in Bangalore

    ReplyDelete
  8. Good to become visiting your weblog again, it has been months for me. Nicely this article that i've been waited for so long. I will need this post to total my assignment in the college, and it has exact same topic together with your write-up. Thanks, good share.
    Data Science Training in Bangalore

    ReplyDelete
  9. Wonderful blog found to be very impressive to come across such an awesome blog. I should really appreciate the blogger for the efforts they have put in to develop such an amazing content for all the curious readers who are very keen of being updated across every corner. Ultimately, this is an awesome experience for the readers. Anyways, thanks a lot and keep sharing the content in future too.

    360DigiTMG Tableau Course

    ReplyDelete
  10. Highly appreciable regarding the uniqueness of the content. This perhaps makes the readers feels excited to get stick to the subject. Certainly, the learners would thank the blogger to come up with the innovative content which keeps the readers to be up to date to stand by the competition. Once again nice blog keep it up and keep sharing the content as always.

    360DigiTMG Business Analytics Course

    ReplyDelete